AL/HR-TR-1 997-0058 


UNITED  STATES  AIR  FORCE 
ARMSTRONG  LABORATORY 


Transferability  of  Skills:  Convergent,  Postdictive, 
Criterion-Related,  and  Construct  Validation  of 
Cross-Job  Retraining  Time  Estimates 


Michael  J.  Kavanagh 

State  University  of  New  York  at  Albany 
Albany,  New  York  12222 

Charles  E.  Lance  \ 

Organizational  Research  &  Development,  Inc. 

390  Radner  Run 
Lawrenceville,  Georgia  30243 

Larry  O’Brien 

Dynamics  Research  Corporation 
60  Frontage  Road 
Andover,  MA  01810 

Robert  B  Stennett 
Robert  McMillen 
Andrew  L.  Solomonson 

The  University  of  Georgia 
Athens,  Georgia  30602 


Doris  E.  Black 

HUMAN  RESOURCES  DIRECTORATE 
COGNITION  and  PERFORMANCE  DIVISION 
7909  Lindbergh  Drive 
Brooks  AFB,  TX  78235-5352 


October  1997 


Approved  for  public  release;  distribution  is  unlimited. 


AIR  FORCE  MATERIEL  COMMAND 
ARMSTRONG  LABORATORY 
HUMAN  RESOURCES  DIRECTORATE 
7909  Lindbergh  Drive 
Brooks  Air  Force  Base,  TX  78235-5352 


XHIC  QUALITY  SHSEEOTBBj* 

ftp  0i'6S-6W 


19990406  090 


NOTICES 


When  Government  drawings,  specifications,  or  other  data  are  used  for  any  purpose  other  than  in 
connection  with  a  definitely  Government-related  procurement,  the  United  States  Government  incurs  no 
responsibility  or  any  obligation  whatsoever.  The  fact  that  the  Government  may  have  formulated  or  in  any 
way  supplied  the  said  drawings,  specifications,  or  other  data,  is  not  to  be  regarded  by  implication,  or 
otherwise  in  any  manner  construed,  as  licensing  the  holder,  or  any  other  person  or  corporation;  or  as 
conveying  any  rights  or  permission  to  manufacture,  use,  or  sell  any  patented  invention  that  may  in  any 
way  be  related  thereto. 

The  Office  of  Public  Affairs  has  reviewed  this  paper,  and  it  is  releasable  to  the  National 
Technical  Information  Service,  where  it  will  be  available  to  the  general  public  including  foreign  nationals. 

This  paper  has  been  reviewed  and  is  approved  for  publication. 


DORIS  BLACK  R-  BRUCE  GOULD 

Project  Scientist  Technical  Director 

Cognition  and  Performance  Division 


WILLIAM  E.  ALLEY 
Chief 

Cognition  and  Performance  Division 


Please  notify  this  office,  AL/HRPP,  7909  Lindbergh  Drive,  Brooks  AFB  TX  78235-5352,  if  your  address 
changes,  or  if  you  no  longer  want  to  receive  our  technical  reports.  You  may  write  or  call  the  STINFO 
office  at  DSN  240-3877  ( or  240-2295)  or  commercial  (210)  536-3877  (or  536-2295). 


U  WhMML 


copies  are  being  forwarded,  indicate  whether  Statement  A.  B.  C.  D.  E.  F.  or  X  applies. 


DISTRIBUTION  STATEMENT  A: 

APPROVED  FOR  PUBLIC  RELEASE:  DISTRIBUTION  IS  UNLIMITED 


Q  DISTRIBUTION  STATEMENT  B: 

DISTRIBUTION  AUTHORIZED  TO  U.S.  GOVERNMENT  AGENCIES 
ONLY;  (Indicate  Reason  and  Date).  OTHER  REQUESTS  FOR  THIS 
DOCUMENT  SHALL  BE  REFERRED  TO  (Indicate  Controlling  DoD  Office). 

EH  DISTRIBUTION  STATEMENT  C: 

DISTRIBUTION  AUTHORIZED  TO  U.S.  GOVERNMENT  AGENCIES  AND 
THEIR  CONTRACTORS;  (Indicate  Reason  and  Date).  OTHER  REQUESTS 
FOR  THIS  DOCUMENT  SHALL  BE  REFERRED  TO  (Indicate  Controlling  DoD  Office). 

[H  DISTRIBUTION  STATEMENT  D: 

DISTRIBUTION  AUTHORIZED  TO  DoD  AND  U.S.  DoD  CONTRACTORS 
ONLY;  (Indicate  Reason  and  Date).  OTHER  REQUESTS  SHALL  BE  REFERRED  TO 
(Indicate  Controlling  DoD  Office). 

D  DISTRIBUTION  STATEMENT  E: 

DISTRIBUTION  AUTHORIZED  TO  DoD  COMPONENTS  ONLY;  (Indicate 
Reason  and  Date).  OTHER  REQUESTS  SHALL  BE  REFERRED  TO  (Indicate  Controlling  DoD  Office). 

[H  DISTRIBUTION  STATEMENT  F: 

FURTHER  DISSEMINATION  ONLY  AS  DIRECTED  BY  (Indicate  Controlling  DoD  Office  and  Date)  or  HIGHER 
DoD  AUTHORITY. 


D  DISTRIBUTION  STATEMENT  X: 

DISTRIBUTION  AUTHORIZED  TO  U.S.  GOVERNMENT  AGENCIES 
AND  PRIVATE  INDIVIDUALS  OR  ENTERPRISES  ELIGIBLE  TO  OBTAIN  EXPORT-CONTROLLED 
TECHNICAL  DATA  IN  ACCORDANCE  WITH  DoD  DIRECTIVE  5230.25,  WITHHOLDING  OF 
UNCLASSIFIED  TECHNICAL  DATA  FROM  PUBLIC  DISCLOSURE,  6  Nov  1984  (Indicate  date  of  determination). 
CONTROLLING  DoD  OFFICE  IS  (Indicate  Controlling  DoD  Office). 

□  This  document  was  previously  forwarded  to  DTIC  on _ (date)  and  the 

AD  number  is _ . 


□ 

□ 

□ 


In  accordance  with  provisions  of  DoD  instructions,  the  document  requested  is  not  supplied  because: 


It  will  be  published  at  a  later  date.  (Enter  approximate  date,  if  known). 
Other.  (Give  Reason) 


DoD  Directive  5230.24,  “Distribution  Statements  on  Technical  Documents,”  18  Mar  87,  contains  seven  distribution  statements,  as 
described  briefly  above.  Technical  Documents  must  be  assigned  distribution  statements. 


Print  or  Type  Name 

<3^0  -3^7-7 _ 

Telephone  Number 


REPORT  DOCUMENTATION  PAGE 

Form  Approved 

OMB  No.  0704-0188 

Public  reporting  burden  forthis  collection  of  information  is  estimated  to  average  1  hour  per  response,  including  the  time  for  reviewing  instructions,  searching  existing  data  sources,  gathering 
and  maintaining  the  data  needed,  and  completing  and  reviewing  the  collection  of  information.  Send  comments  regarding  this  burden  estimate  or  any  other  aspect  of  this  collection  of 
information,  including  suggestions  for  reducing  this  burden,  to  Washington  Headquarters  Services,  Directorate  for  Information  Operations  and  Reports,  1215  Jefferson  Davis  Highway,  Suite 
1204,  Arlington,  VA  22202-4302,  and  to  the  Office  of  Management  and  Budget  Paperwork  Reduction  Project  (0704-0188),  Washington,  DC  20503. 

1.  AGENCY  USE  ONLY  (Leave  blank)  2.  REPORT  DATE  3-  REPORT  TYPE  AND  DATES  COVERED 

October  1997  Final  July  1993-  February  1996 

4.  TITLE  AND  SUBTITLE 

5.  FUNDING  NUMBERS 

Transferability  of  Skills:  Convergent,  Postdictive,  Criterion-Related,  and  Construct 
Validation  of  Cross-Job  Retraining  time  Estimates 

C  -  F41624-92-C-5007 

PE-62205F 

^avanaglC Michael  J.  Stennett,  Robert  B.  Black,  Doris  E. 

Lance,  Charles  E.  McMillen,  Robert 

O'Brien,  Larry  Solomonson,  Andrew  L. 

PR  - 1123 

TA-A7 

WU-04 

7.  PERFORMING  ORGANIZATION  NAME(S)  AND  ADDRESS(ES) 

8.  PERFORMING  ORGANIZATION 

State  University  of  New  York  at  Albany,  Albany,  NY  12222 

Organizational  Research  &  Development,  Inc.,  390  Radner  Run 

Lawrenceville,  Georgia  30243 

Dynamics  Research  Corporation  60  Frontage  Road,  Andover,  MA  01810 

University  of  Georgia,  Athens,  Georgia  30602 

9.  SPONSORING/MONITORING  AGENCY  NAME(S)  AND  ADDRESS(ES) 

10.  SPONSORING/MONITORING 

Armstrong  Laboratory  (AFMC) 

Human  Resources  Directorate 

Cognition  and  Performance  Division 

7909  Lindbergh  Drive 

Brooks  AFB  TX  78235-5352 

AL/HR-TR-1997-0058 

11.  SUPPLEMENTARY  NOTES 

Armstrong  Laboratory  Technical  Monitor:  Doris  E.  Black,  (210)  536-3551 

12a.  DISTRIBUTION/AVAILABILITY  STATEMENT 

12b.  DISTRIBUTION  CODE 

Approved  for  public  release;  distribution  is  unlimited. 

13.  ABSTRACT  (Maximum  200  words)  . 

The  purpose  of  this  project  was  to  validate  estimates  of  cress-job  transferability  of  skills  among  enlisted  speciality  in  the  US 
Air  Force  (USAF).  A  sample  of  retrainees,  their  supervisors  and  peers,  and  a  sample  of  nonretrainees  were  surveyed  on  a 
Transferability  of  Skills  Questionnaire  (TSQ),  which  included  that  (a)  cross-job  retraining  time  estimates  (XJRTEs) 
exhibited  significant  convergent  validity  with  other  AFS-level  predictors  of  retraining  ease  (job  learning  difficulty  and 
cross-AFS  differences  in  aptitude  requirements),  (b)  XJRTEs  exhibited  some  postdictive  validity  when  evaluated  against 
Airman  Retraining  Program  Survey  retraining  ease  criteria,  (c)  XJRTEs  exhibited  significant  criterion-related  validity 
against  time-to-proficiency  (but  not  job  performance)  criteria,  and  (d)  although  retraining  climate  predicted  some  aspects  of 
retraining  success,  it  did  not  consistently  moderate  the  XJRTE-training  success  relationship  as  was  predicted.  Results  are 
discussed  in  terms  of  implications  for  USAF  restructuring  and  retraining  policy. 


14.  SUBJECT  TERMS 
Cross-job  retraining 
Retraining  assignment 
Retraining  time 


Skills  transfer 
Task  taxonomy 
Transfer  of  training 


17.  SECURITY  CLASSIFICATION 
OF  REPORT 

Unclassified 

NSN  7540-01-280-5500 


18.  SECURITY  CLASSIFICATION 
OF  THIS  PAGE 

Unclassified 


15.  NUMBER  OF  PAGES 
_ 143 

16.  PRICE  CODE 

19.  SECURITY  CLASSIFICATION  20.  LIMITATION  OF 
OF  ABSTRACT  ABSTRACT 

Unclassified  UL 

Standard  Form  298  (Rev  2-89)  Prescribed  by  ANSI  Std  Z-39-18 
298-102  COMPUTER  GENERATED 


TABLE  OF  CONTENTS 


Page 


INTRODUCTION  .  1 

General  Background  .  1 

Specific  Background  for  Transferability 

Of  Skills  Project  .  2 

Project  Goals,  Hypotheses,  and  Tasks  .  4 

LITERATURE  REVIEWS . '• . 6 

Transferability  of  Skills  .  6 

Organizational  Structure  .  7 

Transfer  of  Training  .  9 

Implications  for  Research  Project  .  10 

Project  Summary  and  Methodological  Approach  . . 11 

METHOD . 12 

Survey  Development  and  Variable  Measurement  .  12 

Background . 12 

Survey  Development . 13 

Survey  Content  .  13 

Sample . 17 

Data  Editing . 18 

Data  Merging . 18 

Selection  of  Data  Analysis . 19 

Validation  of  Revised  Task  Taxonomy  .  22 

Measurement  of  Retraining  Climate  .  24 

Calculation  of  XJRTEs  .  28 

Construction  of  Input  Data  Sets  for  Analysis . 30 

Convergent  Validation  Data  Set . 30 

Postdictive  Validation  Data  Set . 30 

Construct  and  Criterion-Related 

Validation  Data  Set . 31 

RESULTS . 33 

Convergent  Validation  of  XJRTEs  .  33 

Postdictive  Validation  of  XJRTEs  .  38 

Construct  and  Criterion-Related 

Validation  of  XJRTEs  .  40 

DISCUSSION . 51 

Overall  Project  Objective  .  51 

Tests  of  Hypotheses  and  Completion 


Preceding  Page  Blank 


iii 


Of  Task  Activities . 51 

Implications  for  the  Air  Force . 53 

Cross-Job  Retraining  Time  Estimates  Methodology  .  53 

Organizational  or  Unit  Climate . 55 

Cross -Level  Issues  and  Applications  .  56 

Implications  for  the  Scientific  Community  .  59 

Conclusions . 

REFERENCES . ' . 61 

APPENDIX  A  Transfer  of  Skills  Questionnaires  .  69 

APPENDIX  B  Revised  Taxonomy  Category  Definitions  \ 

Based  on  29  Component  Solution . •  ••  131 

LIST  OF  TABLES 

Table  Pa9e 

1  Number  of  Respondents  for  All  AFSCs  Surveyed . 19 

2  Number  of  Respondents  for  AFSCs  Selected  for  Study  .  21 

3  TSQ  Task  Category  "Part  of  Job"  Rating 

Percentages  with  Representative  AFSCs  .  23 

4  Spearman-Brown  Corrected  Mean  Pairwise 

Interrater  Correlations  .  25 

5  Factor  Analysis  Results  of  Organizational  Climate 

Items  by  Rating  Source  . . 27 

6  Convergent  Validation  -  Descriptive  Statistics 

and  Correlations  with  Cross-Job  Retraining 

Time  Estimates . 33 

7  Convergent  Validation  -  Analysis  of  Variance  Results 

For  Cross -Job  Retraining  Time  Estimates  .  34 

8  Convergent  Validation  —  Analysis  of  Variance  Cell 

Means  (and  Number  of  Observations)  for  XJRTEs 

Based  on  Supervisor  Ratings . 36 

9  Convergent  Validation  -  Moderated  Regression  Results  .  37 

10  Postdictive  Validation  -  AFSs  Surveyed  and  Number  of 

Research  Participants  .  39 

11  Postdictive  Validation  -  Correlations  Between  XJRTEs 

And  ARPS  Criteria . - . 40 

iv 


12  Transferability  of  Skills  Measurement  Model  - 

Factor  Loadings . 44 

13  Transferability  of  Skills  Measurement  Model  - 

Factor  Correlations  .  46 

14  Tests  of  XJRTE  x  Retraining  Climate  Interactions  in 

Predicting  TTP . 49 

15  XJRTES  Example  for  Eight  AFSs  Using  Nonretrainee  and 

Retrainee  MTP  Profiles  .  54 

\ 

LIST  OF  FIGURES 

Figure  Pa9e 

1  Calculation  of  Cross -Job  Retraining  Time  Estimates  .  29 

2  Interaction  Effect  of  From-  and  To-AFS  Aptitude 

Areas  on  Supervisor-Based  XJRTEs  .  36 

3  Similarity  of  From-  vs.  To-AFS  Aptitude  Area  as  a 

Moderator  of  the  Relationship  Between  To-AFS 

OLDs  and  XJRTEs . 37 

4  MTMM  Approach  to  Analysis  of  Relationships  Between 

Retraining-Related  Constructs  and  Method  Factors  ....  41 

5  Situational  Constraint  (SITCON)  x  XJRTE  Interaction 

Effect  on  Decomposed  TTP . 50 


v 


PREFACE 


This  research  and  development  effort  was  conducted  under 
Contract  No.  F41624-92-C-5007  Job  Structuring  Technology.  The 
authors  thank  Bruce  Gould,  Malcolm  Ree,  and  Jacobina  Skinner  for 
their  technical  assistance  and  helpful  comments  at  various  stages 
in  this  project. 


vi 


TRANSFERABILITY  OF  SKILLS;  CONVERGENT ,  POSTDICTIVE, 
CRITERION - RELATED ,  AND  CONSTRUCT  VALIDATION  OF 
CROSS -JOB  RETRAINING  TIME  ESTIMATES 


INTRODUCTION 
General  Background 

The  Armstrong  Laboratory  (AL)  has  been  engaged  in  a  v 
large-scale  research  project  concerned  with  the  estimation  of  the 
manpower,  personnel,  and  training  needs  in  new  and  modified 
weapons  systems,  and  the  support  of  strategic  Pentagon  decisions 
for  existing  weapon  systems .  All  of  these  concerns  have  been 
driven  by  the  need  to  accomplish  mission  requirements  with 
reduced  troop  strength  compared  to  earlier  years. 

Technology  may  assist  the  Air  Force  in  reaching  desired 
troop  strength  reductions.  As  recent  international  clashes 
(e.g.,  the  Gulf  .War)  have  shown,  technology  sophistication  is  of 
paramount  importance  in  the  battle  field  for  all  the  U.S.  forces, 
but  most  particularly  for  the  U.S.  Air  Force  (USAF) .  The  impact 
of  this  demand  for  sophisticated  technology  is  the  creation  of 
new,  or  at  least  expanded,  jobs  in  the  USAF.  Thus,  new  jobs, 
particularly  in  the  electronics  field,  must  often  be  created  to 
handle  these  demands.  The  quandary  the  USAF  must  face  is  how  to 
maintain  these  weapon  systems  with  limited  numbers  of  new 
enlisted  personnel  (accessions)  as  well  finding  qualified 
"first-termers"  (re-enlistees)  who  can  be  transferred  to  these 
jobs. 


A  second  but  equally  important  issue  is  how  to  retain  the 
brightest  of  the  airmen  who  continue  beyond  their  first  term  of 
enlistment.  Some  of  these  fit  into  the  technology  transfer 
quandary  described  in  the  previous  paragraph.  Some  do  not. 

Plus,  there  may  be  a  shortfall  of  people  capable  of  filling  the 
new  technology  jobs  for  a  variety  of  reasons:  (a)  some  of  the 

enlisted  personnel  lack  the  skills  and  abilities  to  do  the 
electrical/electronic  tasks  required  in  the  new  technology 
environment;  and  (b)  some  of  these  enlisted  personnel  do  not  want 
to  transfer  since  they  are  in  line  for  promotion  in  their 
specialty.  Furthermore,  as  the  USAF  becomes  more  technology 
intensive,  it  is  expected  to  become  less  people  intensive,  and 
due  to  the  overall  downsizing  of  troop  strength,  there  will  be  a 
shifting  of  enlisted  personnel  across  Air  Force  Specialties 
(AFSs)  for  a  variety  of  reasons.  Given  these  reasons  and 
challenges,  guidelines  are  needed,  based  on  research  results,  to 
help  solve  the  problem  of  reassignment  of  enlisted  personnel. 
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In  general  from  our  literature  review  it  seems  that  changes 
in  organizational  size  in  terms  of  total  employees,  whether  it  is 
an  increase  or  decrease,  invariably  affect  the  organizational 
structure.  Organizational  structure  includes  the  numbers  and 
kinds  of  people  as  well  as  the  management  hierarchy  necessary  to 
accomplish  the  mission  of  the  organization  in  an  effective  and 
efficient  manner.  Thus,  in  instances  involving  changes  of  the 
numbers  of  employees,  due  either  to  increased  or  decreased  demand 
for  services,  re-alignment  of  the  organizational  structure  is 
necessary. 


Specific  Background  for  Transferability  of  Skills  Project 

At  the  heart  of  this  research  project  is  an  attempt  to 
validate  scientifically  a  procedure  to  estimate  the 
transferability  of  skills  among  the  approximately  180  enlisted 
AFSs  that  will  allow  the  creation  of  an  ease -of -movement  matrix 
(EOMM)  that  will  show  the  ease  of  transferring  task  skills  from 
one  AFS  to  any  other.  Central  to  this  project  has  been  the 
development  of  a  procedure  to  estimate  retraining  time  when  an 
airman  transfers  from  one  AFS  to  a  new  one.  The  staffing  and 
training  (or  cross-training)  requirements  created  as  the  USAF. 
acquires  new  technology  or  has  significant  changes  in  force  size 
may  mean  restructuring  of  AFSs,  major  re-assignments  of 
personnel,  or  both.  These  changes  may  result  in  a  need  to 
identify  the  optimum  occupational  classification  structure  to 
meet  the  needs  of  the  USAF.  It  is  therefore  crucial  that  the  Air 
Force  be  able  to  estimate  how  easy  (or  difficult)  it  is  to 
transfer  enlisted  personnel  across  various  AFSs  in  different  job 
families . 

Before  proceeding,  some  definitions  are  needed.  Consistent 
with  the  literature  (Harvey,  1991) ,  a  position  is  the  basic 
structural  entity  of  the  organization,  and  it  represents  a 
collection  of  work  tasks.  A  job  is  an  "organizational, 
abstraction, "  representing  a  collection  of  positions  within  any 
organization.  A  ~iob  family  is  a  "collection  of  jobs  similar 
enough  in  terms  of  their  important  work  activities  to  be  grouped 
together  and  treated  as  interchangeable  for  a  given  personnel . 
purpose  (Harvey,  1991,  p.  79) .  Frequently,  job  families  within 

an  organization  are  also  referred  to  as  career _ fields ,  and  thus, 

we  will  use  these  two  terms  interchangeably.  Finally,  an 
occupational  classification  structure  shows  all  of  the  job 
families  necessary  to  fulfill  the  mission  of  the  organization. 

In  the  USAF,  the  occupational  classification  structure  is 
represented  by  the  system  of  officer  and  enlisted  Air  Force 
Specialty  Codes,  while  a  single  Air  Force  Specialty  (AFS)  defines 
a  specific  job  family  within  the  system.  An  enlisted  job  within 
the  Air  Force  is  thus  defined  by  its  complete  7-character  AFS 
code  (i.e. ,  including  prefix,  skill  level,  and  suffix) . 
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Returning  to  the  specific  research  issue  of  this  project, 
estimation  of  retraining  times  will  result  in  a  large  number  of 
estimated  values,  considering  there  are  approximately  180 
enlisted  AFSs.  The  EOMM  containing  retraining  time  values  has 
also  been  termed  the  "transferability  of  skills  (TOS)  matrix," 
and  these  two  terms  will  be  used  interchangeably  in  this  report. 
This  EOMM  would  contain  values  showing  the  estimated  time  it 
would  take  to  retrain  moving  from  one  APS  to  any  other  included 
in  the  matrix.  This  matrix  would  be  180  X  180  with  the  "FROM" 
AFSs  listed  as  rows  and  the  "TO"  AFSs  as  columns.  The  values 
above  and  below  the  diagonal  of  the  matrix  would  represent  the 
estimated  retraining  time  of  job  transfers  or  enlisted  \ 
reassignments.  Thus,  by  examining  this  matrix,  one  could  see  the 
ease  of  movement  or  transferability  of  task  skills,  as  quantified 
in  terms  of  retraining  time,  from  any  "target"  AFS  to  any 
specified  set  of  other  AFSs.  This  would  allow  one  to  rank  order 
which  one  of  the  set  of  possible  transfers  for  an  airman  would  be 
the  easiest  in  terms  of  transferability  of  task  skills  as 
measured  by  estimated  retraining  time. 

Kavanagh  and  Lance  (1990)  developed  a  questionnaire 
methodology  to  estimate  cross -job  retraining  times,  and  have 
established  convergent  validity  for  this  methodology.  However, 
the  predictive  validity  of  the  estimates  has  not  been 
demonstrated  due  to  lack  of  adequate  on-the-job  criterion  data. 
One  of  the  primary  objectives  of  this  research  is  the  creation  of 
a  data  base  of  on-the-job  criterion  measures  against  which  the 
estimates  of  cross-job  retraining  times  can  be  empirically 
validated. 

From  a  scientific  viewpoint,  this  research  is  important 
because  it  will  produce  a  methodology  that  can  estimate  cross -job 
retraining  times  across  jobs  in  differing  job  families.  Although 
transferability  of  skills  has  received  some  attention  in  the 
literature  (Byrne,  1975;  Downs,  1985;  Fine,  1957a,  1957b;  Fossum, 
Arvey,  Paradise,  &  Robbins,  1986;  Gopher,  Weil,  &  Baraket,  1994; 
Magnum  &  Ball,  1987;  Rumberger,  1981),  there  is  very  little 
empirical  evidence  supporting  a  method  to  estimate 
transferability  across  jobs  in  different  job  families.  There  has 
been  considerable  research  focused  on  transfer,  or  progression, 
within  career  or  job  families,  but  little  across  job  families. 

In  order  to  meet  the  overall  purpose  of  the  TOS  program,  there 
must  be  a  way  to  estimate  ease -of -movement  across  enlisted  AFSs 
in  different  occupational  or  career  fields.  This  need  is 
particularly  salient  for  airmen  completing  their  first  term  and 
being  reassigned  to  a  new  AFS.  These  retrainees  were  the  target 
group  for  this  research.  Next,  we  present  the  specific 
hypotheses,  project  goals,  and  tasks  necessary  to  complete  this 
project. 
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Project  Goals /  Hypotheses#  and  Tasks 

The  overall  and  major  purpose  of  this  research  project  was 
to  provide  criterion-related  validity  for  cross-job  retraining 
times  estimates  (XJRTEs) .  This  was  a  critical  step  for  the  USAF 
to  have  an  operational  TOS  matrix  of  retraining  time  estimates 
for  the  approximately  180  enlisted  AFSs. 

Consistent  with  the  goal  of  helping  to  develop  a  methodology 
to  establish  a  "transferability  of  skills  matrix"  for  enlisted 
AFSs ,  this  research  project  tested  three  general  hypotheses.  The 
hypotheses  were  as  follows: 

s 

HI:  AFS- level  XJRTEs  will  be  related  to  measures  of  time  to 

achieve  proficiency  and  performance  in  the  new  AFS. 

This  hypothesis  is  based  on  previous  research,  to  be  cited 
relating  to  cross- job  retraining  and  transfer  of  training 
which’ indicates  that  (a)  interjob  similarity  facilitates _ cross¬ 
job  retraining  ease,  (b)  old  versus  new  job  differences  in  job 
learning  times  inhibit  cross-job  transferability  of  skills,  and 
(c)  similarity  between  the  pre-training  and  training  environments 
to  the  post- training  environment  facilitates  transfer  to  on-the- 
job  performance  of  learned  behaviors. 

H2 :  Organizational  structures,  policies,  and  climate,  and 

supervisory  and  peer  behaviors  which  are  supportive  of  retraining 
will  be  positively  related  to  criteria  relating  to  time 
to  achieve  proficiency  following  retraining  and  performance  in 
the  new  AFS . 

This  hypothesis  is  based  on  literature  indicating  that 
variations  in  organizational  structure,  climates  which  are 
supportive  of  post -training  transfer,  and  efforts  on  the  part  of 
*  supervisors  and  co— workers  to  support  the  transfer  of 
previously  acquired  knowledges  and  skills,  are  related  to  the 
ease  with  which  retramees  can  adapt  to  the  post— training 
environment  more  readily  achieving  proficiency  in  the  new 
assignment . 

H3:  The  relationship  between  XJRTEs  and  criteria  relating  to 

performance  and  time  to  achieve  proficiency  in  the  new  AFS  will 
be  stronger  within  organizational  settings  that  are  more  (versus 
less)  supportive  of  training  transfer. 

H3  proposes  an  interaction  effect  between  predicted  cross¬ 
job  retraining  ease  and  the  supportiveness  of  the  retraining 
climate  in  the  new  AFS.  The  general  rationale  for  H3  is  that 
organizational  structures  and  climates  that  are  more  supportive 
of  training  transfer  will  permit  the  full  range  of  training 
transfer  to  be  realized  in  the  post -training  environment.  On  the 
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other  hand,  less  supportive  structures  and  climates  in  the  post¬ 
training  environment  will  have  a  detrimental  (restrictive)  impact 
on  retraining. 

As  is  explained  later,  these  three  general  hypotheses  were 
tested  in  a  number  of  ways  in  three  different  data  sets 
corresponding  to  (a)  a  convergent  validation  of  cross-job 
retraining  time  estimates  against  other  predictors  of  retraining 
ease,  (b)  a  postdictive  validation  of  cross-job  retraining  time 
estimates  against  retraining  criteria  obtained  as  part  of  the 
Airman  Retraining  Program  Survey  (Skinner,  1981) ,  and  (c)  a 
construct-  and  criterion-related  validity  study  using  data 
collected  in  1994  from  retrainees,  their  supervisors,  and  peers, 
and  a  sample  of  nonretrainees . 

As  is  evident  from  these  hypotheses,  cross -job  retraining 
time  ease  is  conceptualized  as  involving  both  a  transfer  of  task 
skills  and  unit  or  organizational  climate  factors  that  might  add 
or  impede  the  ease  of  movement .  Because  of  the  inclusion  of 
these  multiple  factors,  the  literature  review  for  this  project 
was  fairly  extensive.  However,  before  examining  this  literature, 
it  will  help  to  understand  better  this  research  project  if  the 
specific  project  tasks  are  described. 

In  order  to  test  these  hypotheses  and  to  complete  the  goals 
of  this  project,  the  research  effort  was  sub-divided  into  four 
tasks.  These  were: 

Task  1 :  Validation  of  common  task  taxonomy  for  all  AFSs . 

Task  2 :  Exploration  of  alternative  methodologies  for  estimating 
cross- job  retraining  times. 

Task  3 :  Linking  of  retraining  time  estimates  to  measures  of 
actual  retraining  time  success. 

Task  4 :  Development  of  cross-level  linkages  for  XJRTEs . 

Task  1  was  necessary  to  obtain  estimates  of  retraining  ease 
across  tasks  within  a  generic  task  taxonomy.  Much  of  the  work  on 
the  development  of  a  common  task  taxonomy  had  been  done  in 
previous  research  (Lance,  Mayfield,  Kavanagh,  &  Gould,  1995) .  In 
terms  of  Task  2,  convergent  validity  evidence  (Lance,  Kavanagh,  & 
Gould,  1993),  as  well  initial  postdictive  validity  evidence 
(Lance,  Mayfield,  &  Gould,  1993)  has  been  established  for  the 
procedure  used  in  this  research  project.  There  were  no 
alternative  methodologies  to  examine  empirically  since  this 
problem  of  transferability  across  job  families  has  not  been  well 
addressed  in  the  empirical  or  conceptual  literature. 

Task  3  was  the  central  purpose  of  this  research  project  and 
the  hypotheses  are  focused  on  this  task.  However,  it  was 
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necessary  to  complete  Tasks  1  and  2  in  order  to  test  the 
hypotheses  properly  and  complete  Task  3 .  Task  4  is  a  more 
complex  undertaking  and  requires  hypothesized,  generalizable 
extensions  of  the  XJRTE  methodology.  Task  4  will  be  covered  xn 
the  "Discussion"  section  of  this  report. 


LITERATURE  REVIEWS 
Transferability  of  Skills 

Transferability  of  skills  has  been  defined  as  "the 
continuous  use  of  acquired  knowledge  and  abilities  when  moving 
from  one  job  to  another"  (Fine,  1957a,  1957b,  p.  938) ,  or  "in 
terms  of  the  ease  which  individuals  trained  to  proficiency  in  one 
job  can  apply  acquired  knowledge  and  skills  in  learning  another 
job"  (Lance,  Kavanagh,  &  Gould,  1993,  p.  68) .  As  has  been  noted 
earlier  by  Lance  and  colleagues  (Lance,  Kavanagh,  &  Gould,  1993; 
Lance,  Mayfield,  &  Gould,  1993;  Lance,  Mayfield,  Gould,  & 

Lynskey,  1991),  there  exist  large  bodies  of  literature  on  topics 
related  to  cross-job  transferability  of  skills  such  as  transfer 
of  training  to  on-the-job  performance,  curriculum  development, 
job  socialization,  human  resource  planning  for  career  mobility, 
and  employee  turnover.  However,  surprisingly  little  literature 
exists  on  the  transferability  of  skills  across  jobs  in  different 
job  families. 

Among  the  few  empirical  studies  on  cross -job  transferability 
of  skills,  Gordon  and  Fitzgibbons  (1982)  and  Gordon,  Cofer,  and 
McCullough  (1986)  found  that  interjob  similarity  related 
positively  to  cross-job  skill  transfer.  However,  they  studied, 
transfer  of  a  limited  range  of  skills  in  a  small  number. of  sewing 
machine  operator  jobs.  Sparrow  (1989)  also  found  that . inter job 
similarity  related  positively  to  skill  transfer,  but  his  findings 
also  were  confined  to  a  small  range  of  experimentally  derived  job 
tasks . 

The  most  comprehensive  set  of  empirical  studies  on  cross- job 
transferability  of  skills  has  been  conducted  in  recent  years  by 
the  USAF.  In  the  first  of  these,  Lance,  Kavanagh  and  Gould 
(1993)  reported  on  a  methodology  for  estimating  cross- job 
retraining  time  across  41  different  AFSs  in  diverse  career 
fields.  This  methodology  involved  systematic  comparisons  of 
profiles  of  job  learning  times  across  a  comprehensive  taxonomy  of 
job  tasks.  They  demonstrated  the  convergent  validity  of  cross¬ 
job  retraining  time  estimates  against  other  predictors  of 
retraining  success,  including  differences  in  job  general  learning 
difficulties  and  aptitude  requirements. 

In  a  second  study,  Lance,  Mayfield,  Gould. and  Lynskey  (1991) 
demonstrated  the  convergent  validity  of  analytically  derived 
cross-job  retraining  estimates  (i.e.,  using  the  methodology 
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developed  by  Lance,  Kavanagh,  &  Gould,  1993)  with  global 
estimates  provided  by  subject  matter  experts  (SMEs) .  These 
findings  supported  the  viability  of  estimating  cross-job 
transferability  of  skills  from  SME  global  judgments.  Third, 
Lance,  Mayfield,  and  Gould  (1993)  demonstrated _ the  validity  of 
cross-job  retraining  time  estimates  in  predicting  retrospective 
measures  of  retraining  success.  Thus,  although  this  was  a 
retrospective  study,  results  supported  the  validity  of  cross-job 
retraining  time  estimates  in  predicting  criteria  related  to 
actual  retraining  ease. 

Finally,  Lance,  Mayfield,  Foster,  Stokes,  and  Mecham  (1991) 
and  Henderson  (1993)  demonstrated  that  the  methodology  described 
by  Lance,  Kavanagh,  and  Gould  (1993)  for  estimating  cross- job 
retraining  times  is  valid  in  civilian,  as  well  as  in  military 
contexts . 

Despite  these  research  findings,  there  still  exists  a  need 
for  a  large-scale  validation  of  cross-job  retraining  time 
estimates  (i.e.,  estimates  of  the  relative  transferability  of 
skills)  across  a  diverse  set  of  jobs  against  criteria  indicative 
of  actual  retraining  ease.  This  was  one  of  the  primary 
objectives  of  the  present  research.  However,  the  importance  of 
this  review  of  earlier  work  is  that  the  methodology  for  deriving 
XJRTEs  has  been  well  established  and  will  be  used  in  this 
research  project.  Next,  we  examine  the  literature  related  to  the 
organizational  structure,  transfer  of  training,  and  unit  and 
organizational  climate  that  could  affect  ease  of  transfer. 
Therefore,  the  next  two  sections  of  this  literature  review  will 
be  concerned  with  identifying  variables  that  can  be  used  to 
measure  the  organizational  and  unit  variables  that  are 
hypothesized  (H2  and  H3)  to  affect  time  to  proficiency  among 
retrainees . 


Organizational  Structure 

Although  the  structure  of  any  organization  exists  on  paper, 
it  has  profound  effects  on  the  people  within  the  organization. 
Most  employees  would  not  be  able  to  identify  the  overall 
structure  of  their  organization  but  they  can  relate  to  the 
climate  or  culture  permeating  their  work  unit.  Certainly,  as 
seen  in  hypotheses  2  and  3  above,  the  structure  of  the 
organization  is  important  to  understanding  ease  of  movement 
across  jobs  in  the  USAF.  Therefore,  an  extensive  review  of  the 
literature  on  organizational  structure  was  completed  for  this 
project.  The  pertinent  aspects  of  this  review  for  this  project 
are  included  here,  and  the  complete  review  is  available  from  the 
first  author. 

In  1947  the  German  sociologist  Max  Weber  introduced  the  term 
"bureaucracy"  to  describe  a  new  organizational  structure  that  was 


7 


becoming  widespread  in  the  modern  world.  Unlike  the  feudal 
states  and  monarchies  that  preceded  them,  bureaucracies  operated 
according  to  hierarchical  chains  of  command  and  established  rules 
and  procedures  spelling  out  the  official  function  of  each  member. 
In  this  manner,  tight  coordination  and  control  was  realized  that 
far  surpassed  any  previous  organizational  structure.  In  Weber's 
view,  this  new  form  was  so  efficient  that  it  would  eventually 
become  dominant . 

What  followed  Weber's  initial  ideas  was  a  variety  of 
attempts  to  find  the  "best"  organizational  structure  in  terms  of 
organizational  effectiveness.  According  to  Robbins  (1993), 
organizational  structures  vary  along  three  main  dimensions: 
complexity,  formalization,  and  centralization. .  In  addition  to 
these  three  main  dimensions,  researchers  have  identified  five 
other  structural  variables  that  impact  effectiveness:  division 
of  labor,  unity  of  command,  authority  and  responsibility,  span, of 
control .  and  departmentalization .  Our  review  showed  that  each  of 
these  aspects  of  organizational  structure  were  important,  and 
thus,  may  be  important  for  our  questionnaire. 

The  literature  review  also  included  contingency  approaches 
to  organizational  structure  (e.g.,  Burns  &  Stalker,  1961,  Davis  & 
Lawrence  1977,  Robbins,  1993)  as  well  as  resource  dependency 
theory  (Pfeffer  &  Salancik,  1978),  economic  theories  (Moe,  1984), 
and  institutional  theory  (Meyer  &  Rowan,  1977;  DiMaggio  &  Powell, 
1983) .  We  also  examined  in  detail  the  literature  on 
organizational  culture  (Martin,  1992;  Schein,  1992;  Smircich; 
1983)  and  the  related  literature  on  total  quality  management 
(Walton,  1986) . 

Several  observations  seem  warranted  from  this  review. . 

First,  organizational  structure,  which  is  the  formal  definition 
of  line  and  staff  authority  relationships,  and  the  occupational 
classification  structure,  which  is  the  arrangement _ of  job 
families  within  an  organization,  are  related.  It  is  not  that  one 
causes  the  other,  but  rather,  that  they  are  mutually  causative. 
Second,  even  though  the  USAF  exists  as  a  bureaucracy,  there  is 
considerable  latitude  within  individual  units,  at  whatever  level 
or  size,  to  implement  many  of  the  features  of  other  structures 
described  in  this  section.  This  leads  to  the  conclusion  that 
there  could  be  considerable  variance  in  the  type  of 
organizational  structure  an  airman  experiences  as  s/he  is 
transferred  from  one  job  to  another  in  a  new  unit.  Third,  within 
a.  given  organizational  structure,  (e.g.,  bureaucracy) ,  there  can 
be  variance  in  the  degree  of  intensity  or  rigidity  in  which  it  is 
applied  across  organizational  units. 

These  considerations  led  us  to  conclude  that  differences  in 
experienced  organizational  structure  could  affect  the  ease  of 
movement  from  one  AFS  to  another.  Underlying  this  conclusion  is 
the  belief  that  organizational  structure  is  a  perceptual 
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experience  as  a  function  of  the  physical  reality.  Thus,  it  is 
the  airman's  perception  of  the  structure  of  the  new  unit  into 
which  he  or  she  has  been  transferred  which  will  affect  ease  of 
movement.  Due  to  these  considerations,  we  decided  to  include 
measures  of  the  perceptions  of  structure  in  the  "climate"  section 
of  the  questionnaire  developed  for  this  project.  This  aspect  of 
the  questionnaire  will  be  discussed  more  fully  in  a  later  section 
of  this  report . 


Transfer  of  Training 

Although  we  saved  this  literature  until  last,  we  believe  it 
is  the  most  important.  Since  this  project  focuses  on  the 
transfer  of  airmen  from  one  AFS  to  a  new  one,  with  some 
intervening  training,  the  transfer  of  training  literature  is  the 
most  pertinent  place  to  look  for  variables  that  might  impact  on 
this  transfer.  Although  it  is  clear  that  transfer  of  training 
and  ease  of  movement  within  the  TOS  matrix  are  different  events, 
it  was  felt  that  some  of  the  processes  would  be  similar.  This 
section  of  this  report  is  based  on  a  lengthy  review  of  the 
transfer  of  training  literature  (Ruge  &  Kavanagh,  1994)  . 

The  problem  of  transfer  has  been  well  known  to  industrial 
and  organizational  psychology  for  quite  some  time  (McGehee  & 
Thayer,  1961) .  Transfer  involves  applying  what  has  been  learned 
in  training  to  the  actual  work  environment.  Since  the  training 
environment  can  differ  multidimensionally  from  the  work 
environment,  the  transfer  problem  is  a  difficult  one  that  has  yet 
to  be  fully  defined.  Yet  many  variables  have  been  identified 
which  could  lead  to  a  more  positive  transfer  of  learned  knowledge 
and  skills  back  on  the  job  (see  Baldwin  &  Ford,  1988;  Goldstein, 
1986;  Tracey,  Tannenbaum,  &  Kavanagh,  1995;  Wexley  &  Latham, 

1981) . 

Much  of  the  past  research  in  training  mainly  focuses  on 
individual  level  variables.  However,  there  is  a  growing  body  of 
research  suggesting  that  transfer  of  training  is  affected  by 
organizational  level  variables  (Clark,  Dobbins,  &  Ladd,  1993; 
Morrison  &  Brantner,  1992;  Roullier  &  Goldstein,  1991; 

Tracey,  1992)  .  This  revelation  necessitates  an  analysis  based  on 
organizational  levels.  Each  organizational  level  requires  a 
different  approach  to  training.  Yet,  as  these  levels  together 
constitute  a  whole  organization,  the  relations  between  these 
training  approaches  must  also  constitute  an  integrated  whole. 
Previous  treatments  of  training  transfer  have  recognized  and 
subsequently  called  for  research  on  the  problem  that  levels  of 
analysis  present  (Baldwin  &  Ford,  1988) .  For  this  review,  we 
have  identified  the  following  levels:  individual,  supervisor, 
work  group,  and  organizational  factors. 
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Viewing  training  as  a  three -stage  process  has  been  commonly 
accepted  in  the  literature  since  McGehee  and  Thayer's  (1961) 
book.  These  three  stages  consist  of  the  pre- training  stage,  the 
training  stage,  and  the  post -training  stage.  Most  of  the 
pre-training  variables  that  have  been  studied  to  date  have  been 
control  variables  such  as  demographic  or  motivational 
characteristics.  Thus,  pre-training  interventions  have  been 
under-utilized  in  previous  treatments  of  training.  Most  of  the 
research  on  the  training  stage  has  focused  on  the  timing  and 
format  of  the  training  itself,  and  how  each  intervention  affects 
individuals . 

Finally,  while  research  on  the  post-training  stage  has  been 
directed  towards  multiple  levels  of  analysis,  it  has  centered  on 
environmental  control  variables  rather  than  interventions.  While 
there  have  been  some  exceptions  to  this  rule  (e.g.,  behavioral 
self -management ,  relapse  prevention) ,  the  vast  majority  of  work 
at  this  stage  of  training  has  been  more  concerned  with  predicting 
transfer  rather  than  manipulating  it.  We  recognize  that 
manipulations  may  not  always  be  cheap  or  easy  to  implement, 
however,  we  also  recognize  that  relying  on  prediction  of  future 
transfer  may  not  help  managers  in  situations  where  transfer  is 
predicted  to  be  poor.  So  while  we  realize  that  measuring  control 
variables  in  order  to  predict  positive  transfer  is  important,  we 
also  realize  that  managers  will  ultimately  need  treatments  to 
ameliorate  negative  transfer  situations. 

Consideration  of  training  as  a  three  stage  process  and  the 
different  organizational  levels  allowed  us  to  focus  the 
literature  search  on  variables  that  could  impact  on  cross -job 
retraining  times  for  retrainees.  This  approach  assumes  that 
transfer  problems  are  different  at  each  level  of  analysis 
(individual,  supervisor,  work  group,  and  organization)  and  at 
each  stage  of  the  training  process  (pre-training,  training,  and 
post-training).  It  should  be  noted  that  ourreview  (Ruge  & 
Kavanagh,  1994)  focused  mainly  on  studies  which  have  been 
conducted  since  Baldwin  and  Ford’s  (1988)  often  cited  review  on 
transfer  of  training.  However,  we  did  examine  the  Baldwin  and 
Ford  review  for  variables  with  potential  use  in  this  research 
project . 


Implications  for  Research  Project 

As  indicated  earlier,  the  review  of  the  literature  on 
transferability  of  skills  enabled  us  to  identify  a  methodology 
for  XJRTEs  (Lance,  Kavanagh,  &  Gould,  1993).  The  reviews  of _ the 
literature  on  organizational  structure  and  transfer  of  training 
helped  us  identify  a  number  of  variables  that  could  be  used  to 
measure  organizational  and  unit  climate/culture  hypothesized  to 
affect  time  to  proficiency  of  retrainees.  These  variables 
included:  performance  constraints,  organization  and  unit 
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structure,  transfer  of  training  climate,  continuous  learning 
climate,  goal  setting,  procedural  and  distributive  justice, 
interaction  with  supervisor  and  co-workers,  performance  feedback, 
and  existence  of  performance  norms.  This  enabled  us  to 
hypothesize  a  large  number  of  variables  that  should  be  included 
in  the  "climate"  section  of  the  questionnaire  developed  for  this 
search  project.  The  final  variables  that  constituted  the  unit 
climate  measure  of  the  research  questionnaire  will  be  discussed 
in  the  Method  section. 


Project  Summary  and  Methodological  Approach 

\ 

Before  proceeding  to  a  detailed  description  of  the  research 
methodology  used,  we  will  first  summarize  the  major  purpose  of 
this  project.  In  addition,  we  will  provide  a  general  description 
of  the  methodological  approach  used  since  it  is  not  the 
traditional  approach  used  in  criterion-related  validity  studies. 

The  overall  and  major  purpose  of  this  research  project  was 
to  provide  criterion-related  validity  for  XJRTEs.  It  is 
important  to  understand  that  we  are  providing  validity  for  the 
use  of  a  specific  methodology  (Kavanagh  &  Lance,  1990)  to 
estimate  cross -job  retraining  times  (XJRTs) .  The  validity  of  any 
methodology  is  always  intimately  tied  to  the  use  of  that 
methodology.  Thus,  this  does  not  mean  that  this  methodology 
would  automatically  be  valid  for  use  in  estimating  training 
requirements,  although  it  may  be  extended  to  that  purpose. 
However,  subsequent  validity  research  would  be  required  before 
the  methodology  for  estimating  XJRTs  could  be  applied  to  a  new 
situation.  The  primary  purpose  of  establishing  the  validity  of 
this  methodology  was  to  meet  the  goal  of  developing  a  means  by 
which  a  "TOS  matrix"  for  enlisted  AFSs  could  subsequently  be 
constructed. 

The  traditional  scientific  approach  to  establishing 
criterion-related  validity  is  to  measure  the  relationship  between 
predictor  variables,  in  this  case  -  XJRTEs,  and  criterion 
variables,  in  this  case,  job  performance  and  retraining  ease 
information.  The  traditional  approach  may  involve  one  or  more 
type  of  both  predictor  and  criterion  information,  but  generally 
only  one  source  of  measurement  for  both  predictors  and  criterion 
data.  For  example,  in  determining  the  criterion-related  validity 
for  the  Armed  Services  Vocational  Aptitude  Battery  (ASVAB)  for 
selection  and  placement  of  USAF  accessions,  the  ASVAB  scores  are 
the  predictors  and  training  school  performance  is  the  criterion 
(Wilbourn  et  al,  1984) .  In  this  research  project,  we  have 
multiple  sources  of  measurement  for  predictor  and  criterion 
variables  as  well  as  for  the  organizational  climate  variables. 

The  research  variables  were  measured  using  responses  from  the 
retrainee,  his  or  her  supervisor,  and  a  co-worker  or  peer 
selected  by  the  supervisor.  This  multiple  measurement  source 
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approach  to  empirical  research  has  been  termed  multitrait- 
multimethod  (MTMM)  (Campbell  &  Fiske,  1959) ,  and  has  been  used  in 
estimating  the  construct  validity  of  measures  (Kavanagh, 
MacKinney,  &  Wolins,  1971).  We  will  briefly  explain  this 
approach  in  general,  and  provide  more  details  in  the  Method 
section . 

The  MTMM  approach  was  created  as  a  research  design  and  a 
procedure  for  operationally  assessing  the  construct  validity  of 
measures  of  research  variables  (Campbell  &  Fiske,  1959).  It 
requires  both  multiple  measures  of  each  variable  or  trait  and 
multiple  traits.  Thus,  for  example,  in  the  Kavanagh  et  al . 

(1971)  study,  there  were  20  job  performance  dimensions  (the 
traits)  being  rated  by  three  methods,  namely,  supervisor, 
subordinate,  and  self  ratings.  This  resulted  in  60  X  60  matrix 
of  correlations  between  method-trait  measures.  Campbell  &  Fiske 
(1959)  provided  judgmental  rules  for  examining  this  matrixin 
order  to  determine  the  construct  validity  of  the  measures  in 
terms  of  their  convergent  and  discriminant  validities.  Kavanagh 
et  al.  (1971)  provided  a  quantitative  approach  using  an  analysis 
of  variance  procedure  (ANOVA)  as  a  way  to  simplify  and  summarize 
the  evidence  for  validity  contained  in  the  matrix.  However,  for 
this  project,  the  procedures  for  estimating  construct  validity 
are  not  the  central  issue,  but  rather  that  by  using  the  MTMM 
approach  in  this  criterion-related  validity  study,  it  is  possible 
to  estimate  and  account  for  measurement  error  in  the  assessment 
of  the  research  variables.  In  this  way,  the  establishment  of 
criterion-related  validity  can  be  better  evaluated,  and  if 
present,  provides  greater  certainty  in  the  value  of  the  empirical 
results . 

Using  the  MTMM  design  in  this  project  provided  a  better  test 
of  the  project's  hypotheses  and  allowed  us  to  make  stronger 
recommendations  about  the  validity  and  usefulness  of  the  XJRTE  . 
procedure.  Further  details  of  the  MTMM  design  will  be  covered  in 
the  Method  section,  but  it  is  important  that  the  reader  bear  in 
mind  that  we  had  three  sources  or  methods  of  measurement  with 
multiple  variables  or  traits  --a  true  MTMM  design. 


METHOD 

Survey  Development  and  Variable  Measurement 


Background 

Survey  development  in  this  project  was  based  on  reviews  of 
relevant  literature  presented  earlier,  including  literature . 
relating  to  organizational  structure,  transferability  of  skills, 
transfer  of  training  and  retraining  climate. 
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Survey  development  was  also  driven  by  conceptual  models 
developed  in  earlier  work  (Lance,  Kavanagh,  &  Gould,  1993)  in 
which  cross-job  retraining  time  estimates  were  modeled  as  a 
function  of  differences  in  jobs'  task  overlap  and  task  learning 
times . 


Survey  Development 

It  is  important  to  understand  the  history  of  this 
"transferability  of  skills  matrix"  project  in  order  to  understand 
the  rationale  for  the  development  of  the  Transferability  of 
Skills  Questionnaire  (TSQ)  in  this  project.  As  will  be  discussed 
later  in  this  section,  the  sampling  plan  included  sending  surveys 
to  (a)  the  retrainee,  who  is  the  target  of  this  research,  (b)  his 
or  her  supervisor,  and  (c)  a  co-worker  (peer)  of  the  retrainee. 

This  triangular  sampling  was  intended  to  capture  both  the 
job  performance  of  the  retrainee  as  well  as  the  organizational 
climate  of  the  unit.  Rather  than  describing  each  questionnaire, 
we  will  discuss  the  sections  of  the  survey  sent  to  the  retrainee, 
with  reference  to  the  parts  that  were  included  in  the  supervisor 
and  co-worker  surveys.  Copies  of  the  Retrainee,  Supervisor/Peer, 
and  Nonretrainee  versions  of  the  TSQ  are  included  at  Appendix  A. 

The  retrainee  survey  was  designed  to  include  (a)  one  item 
measuring  months  in  the  present  assignment;  (b)  the  measurement 
of  transferability  of  skills  used  in  earlier  research  (Lance, 
Kavanagh,  &  Gould,  1993) ;  (c)  measures  of  the  actual 

time-to-prof iciency  of  the  retrainee;  (d)  measures  of  the  current 
job  performance  of  the  re trainee;  and  (e)  measures  of  the 
organizational  climate  of  the  unit  in  which  the  retrainee  was 
located. 

The  literature  searches  described  earlier  in  this  report 
identified  variables  that  could  potentially  impact  on  retraining 
ease.  The  potential  list  of  variables  was  reduced  by  examining 
the  validity  and  importance  of  each  variable  and  its  measurement . 
An  additional  practical  goal  was  to  limit  the  surveys  to  a 
manageable  length. 


Survey  Content 

Biographical  Information.  Various  computer  files  containing 
biographical  information  on  the  retrainees  (e.g..  Uniform  Airman 
Record  data  containing  aptitude  scores,  data  relating  to  training 
success,  demographic  data,  performance  review  and  base  assignment 
history)  were  made  available  for  this  research  project.  From  a 
lengthy  list  of  potential  personal  variables,  we  selected 
information  such  as  age,  gender,  family  status,  etc.,  as  wellas 
data  on  abilities  and  training  such  as  ASVAB  scores  and  training 
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score  experiences,  including  final  technical  school  grades, 
number  of  hours  of  remedial  instruction,  etc.  The  only 
additional  information  we  collected  from  the  survey  was  the 
length  of  time,  in  months,  the  retrainee  had  been  in  the  present 
assignment . 

AFS  Skills/Knowledges.  This  section  of  the  survey  was 
designed  along  similar  lines  as  the  Skills  and  Knowledges 
Questionnaire  (SKQ)  used  in  earlier  research  (Lance,  Kavanagh,  & 
Gould,  1993;  Lance,  Mayfield,  &  Gould,  1993).  The  SKQ  described 
AFSs '  task  contents  and  learning  times  in  reference  to  a  common 
taxonomy  of  tasks  performed  by  enlisted  airmen. 

Since  the  late  1980s,  Lance  and  Kavanagh  and  associates  have 
conducted  a  stream  of  research  aimed  at  developing  a  task 
taxonomy  specifically  for  the  purpose  of  USAF  R&D  on  cross-job 
transferability  of  skills.  In  one  study,  Lance,  Mayfield, 
Kavanagh,  and  Gould  (1995)  reviewed  the  literature  on  existing 
work  taxonomies  and  described  the  development  of  a  task  taxonomy 
to  support  R&D  on  transferability  of  skills  which  was  based  on 
(a)  previous  USAF  job  categorization  work  (Bell  &  Thommason, 
1984),  (b)  job  analysis  data  collected  from  over  three  thousand 

subject  matter  experts,  and  (c)  previous  work  taxonomy 
literature.  They  also  demonstrated  the  taxonomy's  reliability  in 
terms  of  interrater  reliability  of  SME  judgments  of  task  category 
learning  times,  and  its  validity  in  forming  job  clusters  having 
homogeneous  ability  requirements.  Kavanagh  and  Lance  (1990)  also 
demonstrated  the  usefulness  of  this  taxonomy  for  SME  allocation 
of  tasks  to  task  categories,  and  Lance,  Mayfield,  and  Gould 
(1993)  demonstrated  its  usefulness  for  supporting  cross- job 
retraining  time  estimates.  Nevertheless,  review  of  this  taxonomy 
indicated  the  need  for  its  revision  along  the  lines . of  increased 
specificity  in  the  description  of  more  highly  technical  tasks 
(i.e.,  electronic-  and  mechanically-oriented  tasks). 

With  this  in  mind,  the  taxonomy  was  revised  to  develop  more 
precise  specifications  of  the  domain  of  mechanical  and  electronic 
tasks  thought  to  be  critical  in  retraining  across  more  highly 
specialized  technical  AFSs.  Briefly,  the  revised  task  taxonomy 
was  developed  according  to  the  following  nine  steps: 

(a)  A  complete  listing  of  all  enlisted  Air  Force  Specialty  codes 
(AFSCs)  and  titles  was  obtained  in  order  to  identify  those 
AFSs  which  had  Mechanical  and/or  Electronic  (M/E)  entry- 
level  aptitude  requirements,  or  which  might  otherwise  be 
primarily  mechanically-  or  electronically-oriented.  These 
are  referred  to  as  M/E  AFSs . 

(b)  Occupational  Survey  duty  areas  (titles  of  collections  of 
similar  tasks  which  are  grouped  together  for  job  analysis 
purposes)  were  obtained  for  each  of  the  M/E  AFSs  defined  in 
step  (a) .  The  goal  here  was  to  identify  task  content 
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descriptors  of  intermediate  specificity,  that  is,  more 
general  than  the  task  level,  but  more  specific  than  the 
level  of  general  job  dimensions,  for  the  purpose  of 
generating  generic  task  statements  for  a  multiple-AFS  work 
survey . 

(c)  Duty  area  descriptions  were  translated  into  generic, 
intermediate-level  task  statements  intended  to  be  applicable 
across  a  number  of  M/E  AFSs .  Some  duty  areas  were  judged 
too  broad  to  be  appropriate  for  translation  (e.g., 

"training,"  "supervision")  while  others  were  excluded  from 
retranslation  on  the  basis  that  they  referred  to  specific 
pieces  of  equipment  or  specific  actions  taken  toward  \ 
specialized  equipment.  A  total  of  314  generic  task 
statements  was  written. 

(d)  These  task  statements  were  reviewed  and  edited  for 
redundancy,  readability,  generality  (i.e.,  across  several  of 
the  targeted  AFSs) ,  and  toward  comprehensiveness  in 
representing  the  domain  of  work  performed  by  the  M/E  AFSs 
whose  Occupational  Survey  duty  areas  had  been  obtained.  A 
total  of  180  task  statements  was  retained  after  editing. 

(e)  The  180  edited  task  statements  were  assembled  into  a 
Classification  System  Inventory  (CSI  see  Lance,  Gould, _ 
Stennett,  O'Brien,  Solomonson,  McMillen,  Mallard,  Mayfield, 
Stewart,  &  Jadrich,  in  press) . 

(f)  The  CSI  was  mailed  to  over  3,000  supervisors  and 
superintendents  of  5-skill  level  (i.e.,  journeyman) 
incumbents  in  the  M/E  AFSs  in  the  fourth  quarter  of  1993. 
Respondents  were  asked  to  indicate  the  extent  to  which  each 
of  the  180  tasks  was  "Part  of"  a  typical  5-skill  level  job 
in  their  AFS  (1  =  Very  Small  Amount  to  9  =  Very  Large 
Amount) .  A  total  of  2223  respondents  in  93  different  M/E 
AFSs  returned  completed  CSIs. 

(g)  Principal  components  analyses  were  conducted  on  correlations 

among  the  180  items'  "Part  of  Job"  ratings.  Three  decision 
rules  were  used  to  determine  the  number  of  components  to 
retain  and  interpret:  (i)  Cattell's  (1966)  Scree  test,  (ii) 

Horn's  (1965)  parallel  analysis  criterion,  and  (iii)  . 
interpretability  of  the  solution.  Based  on  these  criteria, 
a  2 9 -component  solution  was  determined  to  best  define  the 
domain  of  M/E  tasks  represented  by  the  180  CSI  items. 

(h)  Components  were  interpreted  according  to  the  generic  task 
statements  having  the  highest  component  loadings  toward 
defining  taxons  (i.e.,  taxonomic  categories)  in  a  revised 
USAF  enlisted  task  taxonomy.  Two  components  of  the  29- 
component  solution  were  deleted  as  uninterpretable,  and  two 
others  were  added  rationally  to  address  content  deficiencies 
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(see  Lance  et  al .  ,  in  press) . 

(i)  Taxon  descriptions  were  developed  to  define  tasks  included 
within  resulting  M/E  task  categories  which  dovetailed  with 
the  remaining  task  categories  to  complete  the  revised 
enlisted  task  taxonomy.  The  complete  revised  enlisted  task 
taxonomy  is  shown  at  Appendix  B. 

Development  and  evaluations  of  this  taxonomy  are  more  fully 
described  by  Lance  et  al.  (1995) . 

This  revised  45-category  task  taxonomy  furnished  the  basis 
for  the  portion  of  the  TSQ  which  defined  retrainees'  AFS  \ 
skills/knowledge  requirements.  Retrainees  were  asked  to  make  a 
total  of  five  ratings  with  respect  to  each  of  the  45  task 
categories.  Retrainees 1  supervisors  and  peers  also  made  the  same 
five  ratings  on  corresponding  versions  of  the  TSQ.  As  in 
previous  research,  the  retrainee  was  asked  to  rate  each  task 
category  as  to:  (a)  whether  it  is  part  of  a  5-level  job  for  a 

typical  airman  in  the  retrainee's  AFS;  (b)  how  much  time  is  spent 
on  that  task  category  for  a  typical  5 -level  airman  in  the 
retrainee ' s  AFS  (1  =  Very  small  amount  to  9  =  Very  large  amount) ; 
and  (c)  how  many  months  it  takes  a  newly  assigned,  3 -level, 
airman  to  become  proficient  at  the  5-level  on  this  task  category 
(1  =  0-1  month  to  9  =  9  or  more  months) .  TSQ  instructions  also 
asked  retrainees  to  rate:  (a)  upon  retraining,  how  many  months 

it  took  them  to  become  proficient  (1  =  0-1  month  to  9=9  or  more 
months) ,  and  (b)  their  current  performance  level  (1  =  Very  much 
below  average  to  9  =  Very  much  above  average) ,  on  each  of  the  45 
task  categories. 

Note  that  the  first  three  of  these  ratings  have  as  the 
rating  target  a  typical  airman  in  the  retrainee's  AFS .  The  last 
two  ratings  changed  this  focus  to  the  particular — retrainee 
targeted  for  research  participation  and  whose  name  appeared  as 
part  of  the  research  packet  given  to  the  retrainee's  Supervisor 
and  Peer. 

Our  primary  criterion  for  the  validation  study  described 
more  fully  later  was  the  time  required  by  the  retrainee  to 
achieve  proficiency  in  the  new  AFS  following  retraining.  Thus 
the  fourth  rating  asked  retrainees  to  rate  how  long  it  took  them 
to  become  fully  proficient  (1  =  0-1  month  to  9  =  9  or  more 
months)  on  tasks  included  within  task  categories  endorsed  as 
"part  of"  their  job. 

Since  we  also  were  concerned  with  the  current  job 
performance  of  the  retrainee,  we  asked  them  to  respond,  using  the 
subset  of  the  45  task  categories  they  chose,  in  terms  of  their 
current  task  performance .  Specifically,  the  fifth  rating  asked 
retrainees  to  rate  their  own  current  performance  level  (1  =  Very 
much  below  average  to  9  =  Very  much  above  average)  with  respect 
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to  each  task  category  endorsed  as  "part  of"  their  job. 

Finally,  a  version  of  the  TSQ  that  contained _ only  the  first 
three  task  category  ratings  (i.e.,  part  of  job,  time  spent  and 
months  to  proficiency  for  a  typical  airman)  was  developed  for  the 
nonretrainee  sample. 

Global  Ratings.  This  section  of  the  TSQ  contained  three 
rating  items  patterned  after  global  ratings  developed  as  part  of 
the  Joint  Service  Job  Performance  Measurement  Project  (Hedge  & 
Teachout,  1986).  These  were  overall  ratings  of  the  retrainee's 
Technical  Proficiency  (Global  Technical  Proficiency,  1  =  Never 
meets  acceptable  level  of  proficiency  to  9  =  Always  exceeds 
acceptable  level  of  proficiency) ,  Interpersonal  Proficiency 
(Global  Interpersonal  Proficiency,  1  =  Never  meets  acceptable 
level  of  proficiency  to  9  =  Always  exceeds  acceptable  level  of 
proficiency) ,  and  Time  to  Proficiency  (Global  Time-to- 
Proficiency,  1=0-1  month  to  9  =  9  or  more  months) .  These  three 
ratings  appeared  only  on  the  Retrainee,  Supervisor,  and  Peer 
versions  of  the  TSQ  and  not  the  Nonretrainee  survey. 

Organizational  Climate.  This  section  of  the  TSQ  assessed 
the  organizational  climate  of  the  retrainee's  job.  The 
retrainee,  the  supervisor,  and  the  co-worker  were  asked  to 
respond  to  the  items  in  this  climate  questionnaire,  including: 

(a)  the  learning  environment;  (b)  supportive  organizational 
structure  and  policies;  (c)  encouragement  of  new  ideas  and 
transfer  of  training;  (d)  goal  setting;  (e)  performance 
constraints;  (f)  co-worker  support;  and,  perhaps  most  important, 
(g)  supervisory  behaviors. 


Sample 

Consistent  with  our  triangulation  approach  to  the 
measurement  of  factors  related  to  retraining  ease,  we  targeted 
recent  retrainees,  their  supervisors,  retrainees'  peers,  as  well 
as  a  sample  of  non-retrainees  for  survey  data  collection. 

Initially,  the  identities  of  26,826  retrainees  for  FY  90-94 
were  obtained  from  the  Air  Force  Personnel  Center .  We  decided  to 
focus  on  recent  retrainees  (i.e.,  MAY  92  through  MAR  94)  at  the 
E-3  to  E-6  grade  level,  who  had  retrained  into  AFSs  in  which  at 
least  100  airmen  had  retrained.  Retrainees  in  grades  E-3  to  E-6 
comprise  the  bulk  of  the  retrainee  population;  persons  in  grades 
lower  than  E-3  typically  are  ineligible  for  retraining  and 
persons  in  grades  higher  than  E-6  are  typically  serving  in 
supervisory  positions.  This  led  to  the  identification  of  3,000 
retrainees  who  were  targeted  for  survey  administration.  An 
additional  1,500  nonretrainees  (also  at  the  E-3  to  E-6  level) 
were  identified  from  an  enlisted  master  personnel  file  (i.e.,  the 
Uniform  Airman  Record  (UAR)  file) ,  for  survey  participation. 
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Nonre trainees  were  deliberately  selected  to  represent  AFSCs  from 
which  many  of  the  retrainees  had  retrained. 

Each  targeted  retrainee's  supervisor  was  sent  by  mail  a 
packet  of  materials  containing  (a)  an  instruction  sheet,  (b)  a 
Transferability  of  Skills  Questionnaire  (TSQ)  to  be  completed  by 
the  supervisor,  (c)  a  TSQ  to  be  completed  by  a  peer  of  the 
re trainee,  (d)  a  TSQ  to  be  completed  by  the  retrainee,  (d) 
computer  scanable  answer  sheets  for  each  of  the  survey  booklets, 
and  (e)  return  mail  envelopes.  Each  retrainee's  supervisor  was 
asked  in  the  instructions  to  complete  his/her  version  of  the  TSQ, 
to  distribute  the  retrainee  version  to  the  retrainee  for 
completion,  and  to  distribute  the  Peer  version  of  the  TSQ, to  one 
peer  of  the  retrainee  to  complete .  Each  of  the  TSQs  were . to  be 
returned  separately.  Nonretrainee  questionnaires  were  mailed 
directly  to  targeted  participants  who  were  asked  to  complete  the 
surveys  and  return  them  separately  also. 

Total  sample  sizes  (and  response  rates)  were  as  follows:  937 
Retrainees  (31%) ,  968  Supervisors  (32%) ,  661  Peers  (22%) ,  and  531 
Nonretrainees  (35%) .  With  the  exception  of  the  Peer  sample, 
these  response  rates  are  somewhat  lower  than  that  in  previous 
transferability  of  skills  studies  (i.e.,  50%  to  58%,  see  Lance, 
Kavanagh,  &  Gould,  1993;  Lance,  Mayfield,  &  Gould,  1993) ,  perhaps 
owing  to  the  complexity  of  the  TSQ. 


Data  Editing 

Answer  sheets  were  scanned  and  four  separate  data  sets  were 
prepared,  each  containing  TSQ  survey  responses  along  with  data 
retrieved  from  existing  data  bases  (i.e. ,  UAR  and  Military 
Entrance  Processing  Station  (MEPS)  files) .  These  correspond  to 
data  for  (a)  retrainees,  (b)  retrainees'  supervisors,  (c)  _ 
retrainees'  peers,  and  (d)  the  nonretrainee  sample.  Initially, 
each  data  set  was  edited  to  insure  that  all  data  elements  were, 
correctly  input  and  that  values  of  all  data  elements  fell  within 
expected  ranges  (e.g.,  that  no  "9's"  occur  for  data  fields  which 
should  only  range  between  1  and  5) ,  by  computing  frequency 
distributions  and  descriptive  statistics  for  all  data  fields. 

The  few  "out -of -range"  values  that  were  detected  were  recoded  as 
missing  data. 


Data  Merging 

Next,  all  four  data  sets  were  merged  into  a  single  data  set 
containing  retrainee,  supervisor,  peer,  and  nonretrainee  data. 
Data  were  merged  using  the  case  control  number  coded  on  the 
surveys.  Thus  each  data  record  corresponded  to  either  a  targeted 
retrainee  (and  his/her  supervisor  and  peer  if  complete  responses 
were  obtained)  or  nonretrainee  participant  (in  which  case  no 
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supervisor  or  peer  data  appeared) . 


Selection  of  Data  for  Analysis 

The  number  of  respondents  by  respondent  group  for  all  AFSs 
surveyed  is  shown  in  Table  1.  Some  AFSs  (e.g.,  AFSC  242x0)  had 
no  Nonretrainee  respondents  as  these  are  AFSs  into  which  many 
airmen  retrain  and  thus  Nonretrainees  were  not  targeted  for  data 
collection.  Other  AFSs  (e.g.,  AFSC  304x2)  had  only  Nonretrainee 
respondents  as  these  AFSs  more  often  serve  as  sources  for  airman 
retraining. 


Table  1 

Number  of  Respondents  for  All  AFSCs  Surveyed 


AFSC 

Title  RETR 

PEER 

SUPER 

NONRET 

MAGE 

OLD 

113X0 

Flight  Engineer 

36 

23 

42 

19 

G 

91 

114X0 

Aircraft  Loadmaster 

19 

10 

26 

8 

G 

115 

201X0 

Intelligence  Operations 

22 

22 

23 

17 

G 

102 

201X1 

Target  Intelligence 

4 

3 

3 

G 

102 

205X0 

Electronic  Intelligence 

7 

5 

6 

G 

102 

241X0 

Safety  Specialist 

8 

9 

7 

G 

105 

242X0 

Disaster  Preparedness 

10 

7 

10 

G 

105 

251X0 

Weather  Specialist 

12 

8 

11 

G-E 

83 

271X2 

Operations  Resources  Mgt 

13 

13 

20 

14 

A 

89 

272X0 

Air  Traffic  Control 

38 

27 

38 

11 

G 

98 

274X0 

Command  &  Control 

7 

9 

6 

G 

95 

277X0 

Space  Systems  Operations 

15 

9 

18 

9 

E 

126 

304X2 

Meteorological  &  Navigation  Sys 

12 

E 

123 

304X4 

Ground  Radio  Communications 

18 

19 

18 

E 

123 

304X5 

Television  Systems 

7 

7 

6 

E 

132 

304X6 

Satellite  Communications  Sys 

5 

4 

4 

E 

127 

305X4 

Elec  Computer  &  Switching  Sys 

27 

27 

28 

E 

120 

306X6 

Secure  Communications  Sys 

13 

E 

123 

361X0 

Antenna  Systems 

8 

M 

124 

361X1 

Communcations  Cable  Sys 

15 

12 

14 

10 

M 

119 

391X0 

Maintenance  Data  Sys  Analysis 

12 

8 

8 

G 

107 

392X0 

Maintenance  Scheduling 

12 

9 

12 

17 

G 

89 

452X4 

Tactical  Aircraft  Maintenance 

18 

13 

22 

36 

M 

120 

454X0A  Aerospace  Propulsion,  Jet  Eng 

14 

M 

141 

454X1 

Aerospace  Ground  Equipment 

5 

M-E 

124 

454X6 

Airlift  Elect  &  Envir  Sys 

11 

7 

6 

M-E 

118 

455X1 

Avionics  Guidance  &  Control  Sys 

18 

15 

19 

14 

E 

126 

455X2 

Communication  &  Navigation  Sys 

17 

16 

18 

20 

E 

126 

457X0 

Strategic  Aircraft  Maintenance 

26 

16 

27 

20 

M 

129 

457X2 

Airlift  Aircraft  Maintenance 

31 

23 

33 

14 

M 

130 

462X0 

Aircraft  Armament  Sys 

11 

M-E 

122 

472X4 

Vehicle  Maintenance  Cnl  &  Analy 

7 

9 

16 

13 

A 

80 

491X1 

Comm- Computer  Sys  Operator 

20 

11 

18 

14 

G 

91 

491X2 

Comm- Computer. Sys  Programmer 

65 

46 

64 

15 

G 

91 

493X0 

Comm- Computer  Sys  Control 

19 

13 

19 

E 

126 

496X0 

Comm- Computer  Sys  Ping  &  Prg  Mgt 

17 

8 

19 

17 

G 

91 

19 


Table  1  (continued) 


AFSC 


Title 


RETR  PEER  SUPER  NONRET  MAGE  OLD 


602XX  Freight  Packaging  &  Traffic  Mgmt 

603X0  Vehicle  Operator/Dispatcher 

605X5  Air  Transportation 

645X0  Inventory  Management 

645X1  Materiel  Storage  &  Distribution 

645X2  Supply  Systems  Analysis 

661X0  Logistics  Plans 

672X1  Financial  Management 

672X2  Financial  Services 

674X0  Financial  Analysis 

702X0  Information  Management 

731X0  Personnel  Sys  Management 

732X0  Personnel 

734X0  Social  Actions 

734X0B  Social  Actions,  Substance  Abuse 

751X0  Education  Specialist 

751X1  Training  Systems 

811XX  Security/Law  Enforcement 

881X0  Paralegal 

906X0  Health  Services  Management 


39 

38 

43 

11 

A 

81 

13 

8 

11 

18 

M 

78 

17 

M-A 

96 

11 

A-G 

79 

12 

13 

14 

G 

70 

8 

9 

13 

13 

A 

91 

25 

9 

25 

A 

125 

5 

6 

3 

12 

A 

100 

8 

11 

8 

6 

A 

87 

28 

17 

24 

G' 

91 

49 

20 

46 

A 

67 

4 

3 

2 

G 

80 

57 

33 

65 

11 

A 

80 

28 

11 

18 

8 

A-G 

86 

12 

A-G 

86 

5 

7 

1 

G 

96 

41 

23 

41 

19 

G 

96 

15 

G 

94 

40 

26 

45 

18 

G 

77 

25 

19 

22 

10 

G 

77 

Total  923  661  942  512 


Note.  AFSC  =  Air  Force  Specialty  Code,  RETR  =  Retrainee,  Peer  =  Co¬ 
worker/Peer,  SUPER  =  Supervisor,  NONRET  =  Nonretrainee,  MAGE  =  MAGE  aptitude 
area  assignment,  OLD  =  Occupational  learning  difficulty. 


As  Table  1  shows,  there  were  several  AFSs  from  which 
relatively  few  respondents  returned  TSQs  (e.g.,  AFSC  201x1). 

Since  many  of  the  analyses  reported  below  were  based  on  AFS-level 
aggregate  data,  we  needed  to  insure  that  the  aggregate -level 
indices  were  estimated  reliably.  Earlier,  Kavanagh  and  Lance 
(1990)  showed  that  approximately  10  subject  matter  experts  (SMEs) 
was  a  sufficient  sample  size  to  insure  reliable  aggregate -level 
estimates  based  on  ratings  of  the  type  collected  here.  Thus,  we 
limited  further  analyses  to  those  AFSs  that  had  at  least  10 
Nonretrainee  respondents,  or  an  average  of  10  Retrainee, 
Supervisor,  and  Peer  respondents,  or  at  least  10  raters  from  two 
of  the  Retrainee,  Supervisor,  and  Peer  respondent  groups.  Sample 
sizes  for  AFSs  thus  retained  for  study  are  shown  in  Table  2. 

The  two  right -most  columns  in  Tables  1  and  2  provide 
information  used  in  analyses  described  later  on  each  AFS's 
aptitude  area  assignment  and  occupational  learning  difficulty 
(OLD) .  The  USAF  bases  accession  and  classification  decisions  in 
part  on  four  aptitude  composites  formed  from  ASVAB  subtests: 
Mechanical,  Administrative,  General,  and  Electronics  (MAGE) 
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Table  2 


Number  of  Respondents  for  AFSCs  Selected  for  Study 


AFSC 

Title  RETR 

PEER 

SUPER 

NONRET 

MAGE 

OLD 

113X0 

Flight  Engineer 

36 

23 

42 

19 

G 

91 

114X0 

Aircraft  Loadmaster 

19 

10 

26 

G 

115 

201X0 

Intelligence  Operations 

22 

22 

23 

17 

G 

102 

242X0 

Disaster  Preparedness 

10 

7 

10 

G 

105 

251X0 

Weather  Specialist 

12 

8 

11 

G-E 

83 

271X2 

Operations  Resources  Mgt 

13 

13 

20 

14 

A 

89 

272X0 

Air  Traffic  Control 

38 

27 

38 

11 

G 

98 

277X0 

Space  Systems  Operations 

15 

9 

18 

E\ 

126 

304X2 

Meteorological  &  Navigation  Sys 

12 

E 

123 

304X4 

Ground  Radio  Communications 

18 

19 

18 

E 

123 

305X4 

Elec  Computer  &  Switching  Sys 

27 

27 

28 

E 

120 

306X6 

Secure  Communications  Sys 

13 

E 

123 

361X1 

Communcations  Cable  Sys 

15 

12 

14 

10 

M 

119 

392X0 

Maintenance  Scheduling 

12 

9 

12 

17 

G 

89 

452X4 

Tactical  Aircraft  Maintenance 

18 

13 

22 

36 

M 

120 

454X0A  Aerospace  Propulsion,  Jet  Eng 

14 

M 

141 

4  55X1 

Avionics  Guidance  &  Control  Sys 

18 

15 

19 

14 

E 

126 

4  55X2 

Communication  &  Navigation  Sys 

17 

16 

18 

20 

E 

126 

457X0 

Strategic  Aircraft  Maintenance 

26 

16 

27 

20 

M 

129 

457X2 

Airlift  Aircraft  Maintenance 

31 

23 

33 

14 

M 

130 

462X0 

Aircraft  Armament  Sys 

11 

M-E 

122 

472X4 

Vehicle  Maintenance  Cnl  &  Analy 

7 

9 

16 

13 

A 

80 

491X1 

Comm- Computer  Sys  Operator 

20 

11 

18 

14 

G 

91 

4  91X2 

Comm- Computer  Sys  Programmer 

65 

46 

64 

15 

G 

91 

493X0 

Comm- Computer  Sys  Control 

19 

13 

19 

E 

126 

496X0 

Comm- Computer  Sys  Ping  &  Prg  Mgt 

17 

8 

19 

17 

G 

91 

602XX 

Freight  Packaging  &  Traffic  Mgmt 

39 

38 

43 

11 

A 

81 

603X0 

Vehicle  Operator/Dispatcher 

13 

8 

11 

18 

M 

78 

605X5 

Air  Transportation 

17 

M-A 

96 

645X0 

Inventory  Management 

11 

A-G 

79 

645X1 

Materiel  Storage  &  Distribution 

12 

13 

14 

G 

70 

645X2 

Supply  Systems  Analysis 

8 

9 

13 

13 

A 

91 

661X0 

Logistics  Plans 

25 

9 

25 

A 

125 

672X1 

Financial  Management 

12 

A 

100 

674X0 

Financial  Analysis 

28 

17 

24 

G 

91 

702X0 

Information  Management 

49 

20 

46 

A 

67 

732X0 

Personnel 

57 

33 

65 

11 

A 

80 

734X0 

Social  Actions 

28 

11 

18 

A-G 

86 

734X0B  Social  Actions,  Substance  Abuse 

12 

A-G 

86 

751X1 

Training  Systems 

41 

23 

41 

19 

G 

96 

811XX 

Security/Law  Enforcement 

15 

G 

94 

881X0 

Paralegal 

40 

26 

45 

18 

G 

77 

906X0 

Health  Services  Management 

25 

19 

22 

10 

G 

77 

Total 

840 

582 

882 

468 

Note.  AFSC  =  Air  Force  Specialty  Code,  RETR  =  Retrainee,  Peer  =  Co¬ 
worker/Peer,  SUPER  =  Supervisor,  NONRET  =  Nonretrainee,  MAGE  =  MAGE  aptitude 
area  assignment,  OLD  =  Occupational  learning  difficulty. 
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(Department  of  Defense,  1984) .  Each  enlisted  AFS  is  classified 
as  drawing  primarily  upon  one  (or  more)  of  the  four  MAGE  aptitude 
areas  (Alley,  Treat,  &  Black,  1988) .  Those  aptitude  area 
assignments  are  reported  in  Tables  1  and  2  for  the  AFSs  in  the 
present  study.  OLD  values  in  Tables  1  and  2  are  a  measure  of  the 
general  learning  difficulty  of  enlisted  AFSs  using  methodology 
developed  by  the  USAF  (Mead  &  Christal,  1970;  see  a  review  and 
construct  validation  by  Mumford,  Weeks,  Harding,  &  Fleishman, 
1987)  .  As  of  Oct  93  benchmarked  OLDs,  which  range  theoretically 
between  25  and  250  (Ramage,  1987)  had  been  derived  for  322  AFSs. 


Validation  of  Revised  Task  Taxonomy 

The  major  goal  of  this  study,  the  calculation  and  validation 
of  estimates  of  cross- job  retraining  times,  rests  on  the 
assumption  that  a  valid  taxonomy  is  available  to  support  cross- 
AFS  comparisons  of  task  content  and  learning  times.  Whereas 
earlier  we  briefly  described  the  development  of  the  taxonomy  used 
in  this  study,  its  validity  needed  to  be  established  prior  to  the 
computation  of  cross- job  retraining  time  estimates.  Also  as 
mentioned  earlier,  a  full  description  of  the  taxonomy's 
evaluation  is  presented  elsewhere  (Lance  et  al.,  in  press) . 
However,  two  aspects  of  the  taxonomy's  evaluation  are 
particularly  relevant  here. 

First,  several  of  the  taxonomy's  categories  were  endorsed 
too  infrequently  as  "part-of"  the  job  to  warrant  additional 
analysis.  Table  3  shows  the  percentage  of  Retrainees  (R) ,  Peers 
(P) ,  Supervisors  (S) ,  and  Nonretrainees  (N)  who  endorsed  each 
task  category  as  ''part  of"  the  job.  Note  that  most  of  the 
difference  between  percentages  within  rows  (i.e.,  AFSs)  is 
between  Nonretrainees  and  the  other  three  rater  groups .  For 
example,  relatively  fewer  Nonretrainees  endorsed  the  PERSONNEL, 
ORAL/WRITTEN  COMMUNICATION,  and  PLANNING/PROBLEM  SOLVING 
categories  as  part  of  their  job,  as  compared  to  the  Retrainee, 
Supervisor,  and  Peer  groups.  On  the  other  hand,  Nonretrainees 
relatively  more  often  endorsed  the  PHYSICAL/MANUAL  LABOR, 
PNEUDRAULIC  SYS  INSPECT,  and  PROPULSION  AND  ENGINE  SYS  task 
categories.  This  is  to  be  expected,  since  the  Retrainees, 
Supervisors,  and  Peers  all  were  describing  the  Retrainees'  jobs, 
while  the  Nonretrainees  alone  were  describing  their  own  jobs. 

Table  3  also  lists  for  each  task  category  a  representative 
AFS  that  had  one  of  the  highest  percentages  of  incumbents 
endorsing  the  category  as  "part  of"  the  job,  to  document  the 
taxonomy's  face  validity.  For  example,  and  as  would  be  expected, 
Personnel  Specialists  often  endorsed  PERSONNEL  tasks  as  "part  of" 
their  job;  Health  Services  Management  Specialists  often  endorsed 
MEDICAL  -  PATIENT  CARE,  and  Weather  Specialists  often  endorsed 
the  task  category  METEOROLOGICAL  EQUIPMENT  MAINT.  Note  however 
that  many  of  the  task  categories  were  infrequently  endorsed, 
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Table  3 


TSQ  Task  Category  MPart  of  Job"  Rating  Percentages  with  Representative  AFSCs 


Task  Category 


Respondent  Representative  AFSC  with  High 

Group  Pctg  Percentage  "Part  of  Job"  Rating 


R  p  S  N  AFSC  Title 


Clerical 

89 

89 

Personnel 

44 

43 

Maintain  Inventories/Records 

55 

55 

Mathematical 

65 

68 

Physical /Manual  Labor 

49 

48 

Manufacturing/Fabricating 

5 

7 

Construction 

3 

4 

Medical  -  Patient  Care 

1 

.9 

Medical  -  Technical 

1 

.5 

Oral /Written  Communication 

87 

89 

Planning/Problem  Solving 

81 

82 

Science/Engineering 

28 

29 

Artistic  -  Audio/Visual 

17 

13 

Food  Preparation 

1 

.8 

Animal  Care 

.2 

.6 

Fabric/Rope  Work 

2 

.9 

Managing  Others 

61 

65 

Training 

75 

78 

Surveillance 

8 

9 

Cryptographic  Equipment  Maint 

6 

6 

Flight  Control /Navig  Sys  Maint  8 

7 

Munitions  Handling 

4 

4 

Test/Meas/Diagnostic  Equip 

5 

6 

Pneudraulic  Sys  Inspect/Maint 

11 

9 

Computer  Sys  Maintenance 

14 

17 

Ground  Based  Comm  Sys  Maint 

5 

6 

Aircraft  Structure  Inspection 

.5 

.5 

Aircraft  Instrumentation  Maint  6 

5 

Meteorological  Equipment  Maint  1 

1 

Aircraft  Armament  Delivery  Sys  2 

1 

Takeoff /Landing  Sys  Maint 

1 

2 

Aircraft  Navig  &  Weapon  Radar 

3 

2 

Malfunction/Recording  Control 

4 

3 

Telephone  Sys  Maint 

10 

9 

Aircraft  Electr  Sys  Maint 

10 

9 

Equip  Circuit  Component  Maint 

2 

1 

Navigational  Aids  Maint 

4 

4 

Satellite/Missile  Sys  Insp 

2 

3 

Airborne  Telemetry  Monitoring 

.6 

.3 

Security  Perimeter  Maint 

.7 

.6 

Avionics  Test  Equip  Maint 

2 

3 

Quality  Control 

22 

25 

Data  Retrieval  &  Storage  Sys 

4 

2 

Space-based  Assets  Control 

3 

3 

Propulsion  and  Engine  Sys 

7 

8 

88 

87 

702x0 

Information  Management 

42 

35 

732x0 

Personnel  Specialist 

53 

59 

645x1 

Materiel  Storage  &  Distr 

63 

64 

113x0 

Flight  Engineer 

48 

58 

645x1 

Material  Storage  Sc  Distr 

6 

10 

361x1 

Communications  Cable  Sys 

3 

7 

361x1 

Communications  Cable  Sys 

.9 

3 

906x0 

Health  Services  Manage 

.5 

.6 

751x1 

Training  Systems 

90 

81 

201x0 

Intelligence  Operations 

84 

76 

272x0 

Air  Traffic  Control 

26 

32 

251x0 

Weather  Specialist 

11 

18 

304x5 

Television  Systems 

2 

2 

114x0 

Aircraft  Loadmaster 

.2 

2 

811x2 

Law  Enforcement 

.8 

3 

114x0 

Aircraft  Loadmaster 

62 

68 

455x2 

Comm  Sc  Navigation  Sys 

73 

79 

242x0 

Disaster  Preparedness 

7 

11 

272x0 

Air  Traffic  Control 

5 

7 

304x6 

Satellite  Comm  Sys  Equip 

9 

12 

455x1 

Avionics  Guidance  Control 

4 

6 

454x6 

Airlift  Elect  &  Envir  Sys 

5 

8 

304x4 

Ground  Radio  Comm 

11 

18 

452x4 

Tactical  Aircraft  Maint 

15 

15 

305x4 

Elec  Computer  Switch  Sys 

5 

6 

304x4 

Ground  Radio  Comm 

.1 

.8 

4  55x2 

Comm  Sc  Navigation  Sys 

6 

9 

455x1 

Avionics  Guidance  Control 

.6 

3 

251x0 

Weather  Specialist 

2 

5 

457X0 

Strategic  Aircraft  Maint 

1 

2 

304X4 

Ground  Radio  Comm 

3 

7 

455x2 

Comm  Sc  Navigation  Sys 

3 

4 

455x1 

Avionics  Guidance  Control 

8 

5 

361x1 

Communications  Cable  Sys 

10 

13 

454x6 

Airlift  Elect  &  Envir  Sys 

1 

3 

454x6 

Airlift  Elect  &  Envir  Sys 

3 

8 

455x2 

Comm  Sc  Navigation  Sys 

2 

4 

4  55x2 

Comm  Sc  Navigation  Sys 

.4 

.4 

205x0 

Elect  Intell  Operations 

.5 

.6 

304x5 

Television  Systems  Spec 

2 

6 

455x1 

Avionics  Guidance  Control 

23 

32 

251x0 

Weather  Specialist 

2 

2 

391X0 

Maint  Data  Sys  Analysis 

3 

2 

277x0 

Space  Sys  Operations 

8 

15 

452x4 

Tactical  Aircraft  Maint 

Note.  R  =  Retrainee,  P  =  Peer,  S  =  Supervisor,  N  =  Nonretrainee . 
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particularly  the  more  specialized  technical  task  categories  which 
were  defined  from  Lance  et  al.'s  (in  press)  revision  of  an  earlier 
task  taxonomy.  This  too  would  be  expected  though,  given  the  goal 
of  increasing  the  specificity  with  which  these  task  categories  were 
defined  in  the  revised  taxonomy.  But  which  categories  had  too  few 
respondents  to  justify  analysis? 


We  used  the  same  rule  of  thumb  discussed  earlier  of  10 
respondents  per  category  to  achieve  sufficiently  reliable  aggregate 
ratings  to  exclude  task  categories  from  further  analysis  due  to 
infrequent  endorsement.  Specifically,  there  was  a  total  sample 
size  of  2,772  among  those  AFSs  retained  for  study  in  Table  2,  or  an 
average  of  693  per  rater  group.  An  average  of  1.4%  part  of  job  ^ 
endorsements  (across  rater  groups)  (i.e.,  1.4%  of  693)  established 
an  approximate  minimum  sample  size  of  10  raters  per  rater  group. 
Thus  we  adopted  the  criterion  of  a  minimum  average  of  1.4%  "part  of 
job"  endorsements  across  rater  groups  to  identify  task  categories 
with  insufficient  endorsement.  As  a  result,  five  task  categories 
(MEDICAL  -  TECHNICAL,  ANIMAL  CARE,  AIRCRAFT  STRUCTURE  INSPECTION, 
AIRBORNE  MONITORING  TELEMETRY,  and  SECURITY  PERIMETER  MAINT)  were 
not  included  in  analyses  reported  below. 


The  other  aspect  of  the  evaluation  of  the  taxonomy  which 
deserves  mention  here  is  interrater  reliability  in  its  use  in 
describing  enlisted  AFSs.  Table  4  shows  Spearman -Brown  corrected 
interrater  reliabilities  (i.e.,  rkks)  for  the  AFSs  retained  for 
analysis.  With  few  exceptions,  the  rkks  are  uniformly  very  high, 
supporting  the  interrater  reliability  of  ratings  with  respect  to 
the  task  taxonomy. 


Measurement  of  Retraining  Climate 

Principal  components  analysis  was  performed  on  the  27  climate 
items.  Separate  analyses  were  conducted  on  each  rating  source: 
supervisor,  peer,  and  retrainee.  Several  criteria  were  used ^ to 
determine  the  number  of  components  to  retain.  First,  Kaiser's 
(1958)  criterion  was  used.  With  this  technique,  components  with 
eigenvalues  greater  than  unity  are  retained.  Second,  scree  plots 
were  examined  for  breaks  in  the  plot  slopes  (Cattell,  1966) .  A 
break  in  the  slope  of  eigenvalues  indicates  the  number  of 
components  to  retain.  Third,  Horn's  (1965)  parallel  analysis  was 
performed.  This  procedure  involves  analyzing  a  randomly  generated 
data  set  with  the  same  structure  as  the  sample  data  set .  The  scree 
plot  for  the  random  data  set  is  superimposed  over  the  scree  plot 
for  the  sample  data.  The  place  where  the  two  scree  plots  intersect 
represents  the  number  of  components  that  should  be  retained. 

Fourth,  interpretability  of  the  components  based  on  the  item 
sources  in  the  literature  (Peters  &  O'Connor,  1980;  Tracey,  1992; 
Tracey  et  al.,  1995)  was  used  as  a  major  criterion  to  determine  the 
final  components  to  measure  organizational  climate. 
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Table  4 


Spearman-Brown  Corrected  Mean  Pairwise  Interrater  Correlations 


AFSC 

Title  Retrainee 

Supervisor 

Peer 

Nonretrainee 

113X0 

Flight  Engineer 

.97 

.97 

.95 

.91 

114X0 

Aircraft  Loadmaster 

.97 

.97 

.96 

—  — 

201X0 

Intelligence  Operations 

.97 

.96 

.97 

.97 

242X0 

Disaster  Preparedness 

.95 

.95 

.95 

—  — 

251X0 

Weather  Specialist 

.95 

.96 

.88 

-  — 

271X2 

Operations  Resources  Mgt 

.95 

.98 

.94 

.  90 

272X0 

Air  Traffic  Control 

.98 

.98 

.82 

.  94 

277X0 

Space  Systems  Operations 

.83 

.95 

.93 

\  — 

304X2 

Meteorological  &  Navigation  Sys 

— 

— 

.  94 

304X4 

Ground  Radio  Communications 

.97 

.97 

.97 

-  — 

305X4 

Elec  Computer  &  Switching  Sys 

.98 

.97 

.98 

— 

306X6 

Secure  Communication  Sys 

— 

-  - 

.  94 

361X1 

Communications  Cable  Sys 

.94 

.96 

.94 

.92 

392X0 

Maintenance  Scheduling 

.  96 

.96 

.93 

.79 

452X4 

Tactical  Aircraft  Maint 

.94 

.97 

.95 

.91 

454X0A 

Aerospace  Propulsion,  Jet  Eng 

-- 

— 

.  92 

455X1 

Avionics  Guidance  &  Control  Sys 

.93 

.92 

.92 

.  92 

455X2 

Communication  &  Navigation  Sys 

.91 

.91 

.89 

.93 

457X0 

Strategic  Aircraft  Maintenance 

.96 

.97 

.94 

.88 

457X2 

Airlift  Aircraft  Maintenance 

.95 

.97 

.95 

.85 

462X0 

Airlift  Armament  System 

-- 

— 

.95 

472X4 

Vehicle  Maintenance  Cnl  &  Analy 

.94 

.98 

.94 

.95 

491X1 

Com- Computer  Sys  Operator 

.96 

.97 

.93 

.94 

4  91X2 

Com- Computer  Sys  Programmer 

.98 

.99 

.97 

.  96 

493X0 

Com- Computer  Sys  Control 

.94 

.  96 

.94 

—  — 

496X0 

Com- Computer  Ping  &  Prg  Mgt 

.97 

.97 

.  96 

.  99 

6  02XX 

Freight  Packaging  &  Traffic  Mgt 

.78 

.96 

.  98 

.  93 

603X0 

Vehicle  Operator/Dispatcher 

.93 

.91 

.90 

.  96 

605X5 

Air  Transportation 

— 

-  - 

.  97 

645X0 

Inventory  Management 

— 

— 

.89 

645X1 

Materiel  Storage  &  Distribution 

.94 

.95 

.95 

-  - 

645X2 

Supply  Systems  Analysis 

.92 

.95 

.  88 

.  94 

661X0 

Logistics  Plans 

.96 

.  98 

.  96 

—  — 

672X1 

Financial  Management 

— 

— 

.  96 

674X0 

Financial  Analysis 

.99 

.98 

.96 

-  - 

702X0 

Information  Management 

.99 

.99 

.98 

—  “ 

732X0 

Personnel 

.99 

.99 

.69 

.  96 

734X0 

Social  Actions 

.98 

.98 

.98 

-  - 

734X0B 

Social  Actions,  Substance  Abuse 

.97 

751X1 

Training  Systems 

.95 

.99 

.98 

.98 

811XX 

Security/Law  Enforcement 

— 

-  - 

.  95 

881X0 

Paralegal 

.99 

.99 

.98 

.97 

906X0 

Health  Services  Management 

.97 

.74 

.  96 

.  90 

Note:  Two  dashes  (--)  denote  cells  with  insufficient  sample  sizes. 
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Based  on  these  criteria,  three  components  were  retained  in  the 
final  analyses.  Thre  three  component  solution  for  the  three 
rating  sources  were  compared,  and  items  common  to  the  three 
components  were  identified  for  inclusion  in  the  final  climate 
scales.  Table  5  contains  the  items  and  component  loadings  for 
the  final  three  scales  for  Supervisors,  Peers,  and  Retrainees. 

The  item  numbers  in  the  table  refer  to  the  original  item  numbers 
of  the  SKQs  in  Appendix  A. 

Since  the  SKQ  items  for  all  three  rating  sources  are  the 
same,  examining  the  items  within  the  components  in  Table  5  for 
the  retrainees  provides  meaning  for  the  three  organizational . 
climate  components  identified  through  these  analyses.  The  first 
component.  Situational  Constraints,  is  characterized  by  factors 
in  the  work  unit  that  can  affect  an  individual  airman|s  job 
performance  like  "a  lack  of  qualified  people  in  my  unit,"  "Too 
much  'red  tape',"  and  "inconsistent  policies,  procedures,  and 
instructions."  This  organizational  climate  variable  is  very 
similar  to  situational  constraints  factors  found  in  earlier 
studies  (Peters  &  O'Connor,  1980).  The  second  organizational 
climate  component,  Organizational  Support  for  Continuous 
Improvement .  is  characterized  by  components  in  the  work  unit  that 
create  a  climate  that  encourages  innovation  and  continuous 
learning.  This  organizational  climate  is  established  by  multiple 
signals  like  "coworkers  are  willing  to  listen  to  new  ideas," 
"coworkers  tell  each  other  about  new  information  that  can  be  used 
to  increase  job  performance,"  "supervisors  openly  express  their 
support  of  continuous  learning  on  the  job, "  and  "my  work  unit  is 
highly  innovative."  This  organizational  climate  variable  is 
quite  similar  to  the  one  found  by  Tracey  et  al.  (1995)  in  their 
study  of  transfer  of  training  in  a  sample  of  supermarket 
managers.  The  third  component.  Supervisor — Support — for — Transfer 
of  Training,  focuses  specifically  on  factors  that  foster  transfer 
of  training  for  the  retrainee.  The  support  in  the  work  unit  for 
transfer  of  training  comes  from  multiple  sources  such  as  "there 
is  excellent  on-the-job  training,"  "supervisors  set  goals  for 
newly  retrained  airmen  which  encourage  them  to  apply  training  on 
the  job, "  and  "training  aids  are  available  on  the  job  to  support 
what  newly  trained  airmen  learned  in  retraining."  Again,  this 
organizational  climate  component  is  similar  to  those  identified 
in  the  previous  literature  (Tracey,  1992;  Tracey  et  al . ,  1995). 

The  internal  consistency  estimates  (Cronbach's  alphas)  for 
the  climate  scales  for  each  source  were:  (a)  Retrainees: 
Situational  Constraints  -  .84,  Organizational  Support  for 
Continuous  Improvement  -  .81,  Supervisor  Support  for  Transfer  of 
Training  -  .72;  (b)  Peers:  Situational  Constraints  -  .85, 

Organizational  Support  for  Continuous  Improvement  -  .77, 
Supervisor  Support  for  Transfer  of  Training  -  .69;  and. (c) 

Supervisors:  Situational  Constraints  -  .85,  Organizational 
Support  for  Continuous  Improvement  -  . 73 , . Supervisor  Support  for 
Transfer  of  Training  -  .68.  Thus,  all  climate  subscales  had 


26 


Table  5 


Factor  Analysis  Results  of  Organizational  Climate  Items  by  Rating  Source 


SUPERVISOR 

PEER 

RETRAINEE 

FACTOR 

TrPT?M 

FACTOR 

TTFM 

FACTOR 

ITEM  - 

NO.  1  2 

3 

X  1 

NO. 

1  2 

3 

NO. 

1  2 

3 

SITUATIONAL 

CONSTRAINTS 

80. 

.71 

-.15 

-  .21 

80. 

.69 

-.26 

-.13 

66. 

.67 

\ 

-.10 

-.27 

71. 

.68 

-.03 

-  .06 

67. 

.68 

-  .23 

-  .08 

70. 

.67 

-  .03 

-  .07 

77. 

.67 

-  .03 

-.18 

71. 

.68 

.02 

-.15 

71. 

.64 

-.06 

-.00 

67. 

.66 

-.07 

-.07 

77. 

.65 

-  .11 

-.00 

78. 

.64 

-.20 

-.22 

82. 

.65 

-.12 

-.09 

82. 

.64 

-.02 

-.24 

85. 

.64 

-.15 

-.12 

74. 

.61 

-  .06 

-  .19 

74. 

.64 

-  .07 

-.31 

65. 

.61 

-  .11 

-.28 

90. 

.61 

.06 

-  .19 

84. 

.60 

.00 

-.26 

79. 

.60 

-.08 

-  .13 

65. 

.60 

-.09 

-  .13 

90. 

.58 

-.08 

-.10 

87. 

.60 

.08 

-.09 

75. 

.59 

-  .19 

.01 

66. 

.58 

-  .31 

-  .07 

67. 

.57 

-.26 

-  .23 

84. 

.51 

-  .15 

-.07 

75. 

.54 

-  .23 

.01 

74. 

.50 

-.16 

-.16 

69. 

.51 

-.27 

-.02 

65. 

.52 

-.31 

-.07 

84. 

.50 

-.22 

-.03 

86. 

.31 

-.25 

.18 

86. 

.28 

-.14 

.17 

72. 

.36 

-.20 

.18 

ORGANIZATIONAL 

SUPPORT  FOR  CONTINUOUS 

IMPROVEMENT 

89. 

-  .14 

.67 

.11 

78. 

-  .23 

.74 

.10 

89. 

-.18 

.74 

.09 

88. 

.02 

.66 

.04 

79. 

-  .09 

.72 

.22 

90. 

-  .20 

.71 

.16 

87. 

-.17 

.63 

.16 

91. 

-.09 

.68 

.29 

83. 

-.07 

.65 

.28 

85. 

-.04 

.60 

.07 

87. 

-.09 

.65 

.15 

91. 

-.20 

.63 

.35 

91. 

-  .17 

.56 

.34 

68. 

-.20 

.61 

.06 

82. 

-  .08 

.60 

.15 

79. 

-.14 

.56 

.33 

73. 

-  .24 

.61 

.14 

81. 

-  .15 

.58 

.28 

73  . 

-.13 

.54 

.19 

89. 

-  .19 

.58 

.18 

92. 

-.23 

.56 

.37 

68. 

-  .12 

.51 

.22 

70. 

-  .08 

.56 

.33 

86. 

-.03 

.50 

.04 

83. 

-  .04 

.54 

.47 

SUPERVISOR 

SUPPORT  FOR  TRANSFER  OF 

TRAINING 

70. 

-.09 

.24 

.67 

81. 

-.23 

.19 

.65 

76. 

-  .28 

.22 

.73 

83. 

-  .13 

.35 

.63 

72  . 

-  .10 

.16 

.61 

77. 

-  .17 

.32 

.70 

72  . 

-  .16 

-.03 

.62 

76. 

-.08 

.09 

.60 

69. 

-.13 

.23 

.69 

66. 

-.15 

.19 

.59 

66. 

-.24 

.23 

.58 

68. 

-.19 

-.01 

.66 

81. 

-.23 

.13 

.56 

88. 

-  .06 

.23 

.57 

80. 

-.12 

.33 

.64 

76. 

.03 

.20 

.54 

85. 

-  .06 

.21 

.55 

73. 

-.00 

.28 

.50 

78. 

-  .17 

.43 

.48 

75. 

-.20 

.42 

.46 

Note:  Item  number  refers  to  the  TSQ  contained  in  the  Appendix  A. 
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acceptable  reliabilities  and  were  consistent  with  the  literature 
sources  used  for  item  selection.  However,  the  one  note  of 
caution  for  these  results  is  that  the  reliabilities  were  based  on 
the  scales  after  taking  advantage  of  the  principal  component 
solutions,  and  thus,  should  be  subjected  to  subsequent 
verification  on  a  new  sample. 


Calculation  of  XJRTEs 

We  used  an  adaptation  of  the  methodology  described  by  Lance, 
Kavanagh,  and  Gould  (1993)  to  derive  cross-job  retraining  time 
estimates  (XJRTEs) .  The  adaptation  was  necessary  due  to  the  AFSs 
represented  by  the  Retrainee  (and  Supervisor  and  Peer)  versus 
Nonretrainee  samples  in  this  study .  As  has  been  shown  earlier 
(e.g.,.  Lance,  Mayfield,  &  Gould,  1993)  retraining  frequencies 
across  various  AFSs  are  asymmetric,  with  some  AFSs  (e.g., 

Security  Specialist  and  many  of  the  mechanical  AFSs)  much  more 
often  serving  as  "sources"  for  retraining,  and  others  (e.g., 
administrative  and  personnel  AFSs)  serving  as  "recipients"  or 
"targets"  for  retraining.  Thus  in  the  present  sample,  the 
Retrainees  (and  Supervisors  and  Peers)  represented  the  "Into- 
AFSs"  in  the  sense  of  retraining  from  one  AFS  into  another,  but 
they  underrepresented  the  "From- AFSs."  On  the  other  hand,  the 
Nonretrainee  sample  was  deliberately  chosen  to  represent  those 
AFSs  that  most  typically  represent  the  source  AFSs  for 
2T©  training ,  ox  th©  ” Firom-AFSs .  n  Thus,  La.nc©,  Ka.va.nagh,  and. 
Gould's  (1993)  profile  comparison  methodology  was  adapted  here  by 
using  Nonretrainee  rating  profiles  to  describe  the  "From-AFSs" 
and  Retrainee,  Supervisor,  and  Peer  rating  profiles  to  describe 
the  "Into-AFSs"  for  the  purpose  of  computing  XJRTEs. 

Specifically,  mean  Month-to- Proficiency  (MTP,  i.e.,  the 
third  task  category  rating  on  the  TSQ)  rating  profiles  were 
computed  for  each  respondent  group  for  each  AFS  across  the  40 
task  categories  retained  for  analysis.  To  compute  cross- job 
retraining  time  estimates,  mean  MTP  profiles  (e.g. ,  XI  and  X2, 
for  AFS1  and  AFS 2 ,  respectively)  were  compared  element-wise 
across  the  40  task  categories  to  calculate:  (a)  sums  of 

differences  between  XI  and  X2  values  for  which  XI  values  are 
larger  (indicating  task  skills  to  be  acquired  in  retraining  from 
AFS2  into  AFS1) ,  and  (b)  sums  of  differences  between  XI  and  X2 
values  for  which  X2  values  are  larger,  indicating  task  skills  to 
be  acquired  in  retraining  from  AFS1  into  AFS2  (see  Figure  1  and 
Lance,  Kavanagh,  &  Gould,  1993;  Lance,  Mayfield,  Gould,  & 

Lynskey,  1991).  Note  that  three  sets  of  XJRTEs  were _ calculated 
on  the  basis  of  profile  comparisons  between  Nonretrainee  MTP  mean 
profiles  representing  the  "From-AFSs"  and  (a)  Retrainee,  (b) 
Supervisor,  and  (c)  Peer  mean  MTP  profiles  each  representing  the 
"Into-AFSs."  Note  also  that  this  methodology  generates  XJRTEs 
which  (a)  reflect  the  cross-job  retraining  times  across  (perhaps 
widely)  different  jobs,  (b)  are  asymmetric  with  respect  to 
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Figure  1.  Calculation  of  Cross -Job  Retraining  Time  Estimates 


retraining  across  different  jobs  (e.g.,  retraining  from  Surgeon 
to  Nurse  would  be  estimated  to  be  easier  than  from  Nurse  to 
Surgeon) ,  and  (c)  are  relative  in  the  sense  that  no  attempt  is 
made  to  estimate  actual  retraining  times. 


Construction  of  Input  Data  Sets  for  Analysis 

As  we  mentioned  earlier,  the  three  general  hypotheses 
outlined  above  were  evaluated  in  a  number  of  ways  using  three 
independent  data  sets  corresponding  to  (a)  a  convergent 
validation  study,  (b)  a  postdictive  validation  study,  and  (c)  a 
construct-  and  criterion- related  validation  study  of  XJRTEs. 

This  section  describes  the  construction  of  these  three  data  sets. 


Convergent  Validation  Data  Set 

The  convergent  validation  data  set  was  designed  to  replicate 
results  presented  earlier  by  Lance,  Kavanagh,  and  Gould  (1993)  at 
the  AFS- level  of  analysis.  Lance,  Kavanagh,  and  Gould  (1993) 
argued  that  XJRTEs  should  be  related  to  two  other  AFS- level 
variables,  occupational  learning  difficulty  (OLD)  and  the  AFS's 
aptitude  area  assignment. 

Lance,  Kavanagh,  and  Gould's  (1993)  hypotheses  were  that 
XJRTEs  (a)  would  relate  positively  to  the  occupational  learning 
difficulty  (OLD)  of  the  "Into-AFS,"  under  the  rationale  that 
£K2tid.gving  proficiency  in  a  new  AFS  should  generally  take  longer 
into  AFSs  that  are  generally  more  difficult  to  learn,  (b) 
generally  should  be  higher  for  retraining  across  AFSs  that  have 
different,  versus  the  same  aptitude  requirements,  and  (c)  that 
the  positive  XJRTE  -  OLD  relationship  should  be  stronger  for 
retraining  movements  across,  versus  within,  aptitude  areas. 

Each  row  of  the  AFS- level  data  set  constructed  to  evaluate 
these  hypotheses  contained  (a)  three  XJRTEs  for  retraining  from 
AFS  to  AFS-  (i  ^  j),  based  on  profile  comparisons  between 
Nonretrainee  mean  MTP  profiles  and  Retrainee,  Supervisor,  and 
Peer  mean  MTP  profiles,  (b)  the  OLD  for  AFS*  (the  "From-AFS"), 

(c)  the  OLD  for  AFSj  (the  "Into-AFS"),  (d)  a  code  number 
indicating  the  MAGE-area  assignment  for  AFSi  (M=l,  A=2,  G=3, 

E=4)  (0)  0  similar  code  indicating  the  MAGE-area  assignment  for 

AFS  ,#  and  (f)  a  binary  variable  ("MAGE-F/T")  indicating  whether 
AFS-  and  AFSj  were  assigned  to  the  same  (  =  0)  or  different  (=1) 

MAGE  aptitude  areas. 


Postdictive  Validation  Data  Set 

The  second  data  set  was  designed  to  replicate  results 
presented  earlier  by  Lance,  Mayfield,  and  Gould  (1993) .  They 
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conducted  a  postdictive  validation  of  XJRTEs  against  criteria 
related  to  retraining  success  that  were  collected  as  part  of  the 
Airman  Retraining  Program  Study  (ARPS)  in  the  late  1970s  (see 
Lance,  Mayfield,  &  Gould,  1993;  Skinner,  1981,  1982,  1983; 

Skinner  &  Alley,  1980,  1984  for  fuller  descriptions  of  the  ARPS). 
ARPS  criteria  included  Retrainee  and  Supervisor  ratings  of  (a) 
the  extent  to  which  the  technical  skills  learned  in  the 
retrainee's  previous  specialty  were  helpful  in  their  new 
(current)  specialty  ("Previous  Skill  Utilization"),  and  (b)  the 
retrainee's  technical  skills  and  abilities  ("Current 
Skill/Ability") .  Also,  Supervisors  rated  the  Retrainee's  current 
amount  and  quality  of  work  performance  ("Performance 
Effectiveness") . 

Recognizing  that  the  enlisted  USAF  specialty  structure  had 
changed  many  times  over  the  last  20  years,  we  tracked  the  AFSCs 
of  the  AFSs  retained  for  analysis  in  this  study  to  determine 
whether  comparable  specialties  existed  at  the  time  of  the  ARPS 
(i.e.,  c.  1978).  Surprisingly,  many  of  the  1994  AFSCs  were 
identical  to  those  in  1978.  Others  had  simply  changed  AFS  codes. 
Still  others  had  changed  so  radically  that  it  was  impossible  to 
identify  1978  AFSCs  that  corresponded  to  1994  AFSCs.  In  the  end, 
we  retained  ARPS  data  only  for  those  retrainees  whose  "From- 
AFSCs "  and  "Into- AFSCs"  were  traceable  to  comparable  AFSCs  in  the 
1994  enlisted  USAF  structure  and  which  were  included  in  the 
present  study.  As  is  reported  more  fully  later  in  Table  10, 

1,940  ARPS  retrainees  with  complete  retrainee  and  supervisor 
criterion  data  were  retained  for  this  analysis. 

Thus,  the  final  postdictive  validation  data  set  consisted  of 
the  three  sets  of  XJRTEs  (i.e.,  based  on  Retrainee,  Supervisor, 
and  Peer  MTP  ratings)  computed  from  the  present  TSQ  data  (which 
were  matched  to  comparable  From-  and  Into-AFS  retraining 
movements  that  had  occurred  in  the  late  1970s)  and  ARPS  criteria 
described  earlier.  Note  that  in  this  data  set  the  XJRTEs  are 
computed  at  the  AFS- level  of  analysis,  whereas  criteria  are 
measured  at  the  individual  retrainee-level  of  analysis. 


Construct  and  Criterion  Related  Validation  Data  Set 

This  data  set,  discussed  in  part  already,  consisted  of  TSQ 
data  collected  specifically  for  this  study  in  1994  from  the 
sample  of  Retrainees,  their  Supervisor  and  a  Peer,  and  from  a 
sample  of  Nonretrainees.  Each  data  record  corresponded  to  a 
Retrainee  or  a  Nonretrainee .  Both  Retrainee  and  Nonretrainee 
records  were  augmented  with  personnel  data  (e.g.,  ASVAB  scores) 
obtained  from  additional  computer  files  (e.g.,  UAR  and  MEPS 
data) .  Retrainees'  records  also  contained  matched  Supervisor  and 
Peer  data  (when  available) ,  as  well  as  AFS-level  data  including: 
(a)  the  three  sets  of  XJRTEs  described  earlier,  (b)  OLDs  of  the 
Retrainee's  previous  and  current  specialty,  and  (c)  codes 
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indicating  the  Retrainee's  previous  and  current  AFSs'  MAGE  area 
assignment.  Five  additional  variables  were  computed  from 
Retrainee,  Supervisor,  and  Peer  TSQ  data  for  analyses: 

"Decomposed"  Time- to-Prof iciency .  Recall  that  in  addition 
to  the  global  estimates  of  retrainee  time  to  achieve  proficiency 
in  the  new  specialty,  Retrainees,  Supervisors,  and  Peers  also 
rated  retrainee  time-to-prof iciency  with  respect  to  each  of  the 
40  task  categories  on  the  TSQ  that  were  relevant  for  the 
Retrainee's  AFS .  We  computed  a  "decomposed"  time-to-prof iciency 
(DTTP)  measure  for  each  rating  source  as  the  simple  sum  of  these 
ratings  across  the  relevant  task  categories  (i.e. ,  those  task 
categories  that  were  endorsed  as  "part  of"  the  job) .  \ 

Conceptually,  DTTP  measures  reflect  a  composite  estimate  of  total 
time  to  achieve  job  proficiency  as  an  additive  function  of 
achieving  proficiency  on  individual  job  elements. 

"Decomposed"  Job  Performance.  The  fifth  TSQ  task  category 
rating  completed  by  Retrainees,  Supervisors,  and  Peers  was  of  the 
Retrainee's  performance  effectiveness  with  respect  to  each 
relevant  task  category.  We  computed  a  "decomposed"  job 
performance  (DPERF)  measure  for  each  rating  source  on  the  basis 
of  these  ratings  as  a  simple  sum  of  the  ratings  across  relevant 
task  categories  (i.e.,  those  task  categories  that  were  endorsed 
as  "part  of"  the  job)  divided  by  the  number  of  task  categories 
endorsed.  Dividing  by  the  number  of  task  categories  endorsed 
guarded  against  the  possibility  of  Retrainees  achieving  higher 
performance  scores  simply  because  more  task  categories  were 
endorsed  as  relevant.  Thus  DPERF  reflects  retrainee  performance 
averaged  across  relevant  task  categories . 

Retraining  Climate  Variables.  Finally,  retraining  climate 
scores  were  calculated  for  each  rating  source  based  on  the 
principal  components  analyses  reported  earlier.  Three  composites 
reflecting  Situational  Constraints.  Supervisor  Support,  and 
Organizational  Support  were  calculated  as  the  mean  score  across 
relevant  scale  items. 

In  summary,  and  consistent  with  the  triangulation 
measurement  approach  mentioned  earlier.  Retrainee,  Supervisor, 
and  Peer  measures  each  were  obtained  for: 


Retraining  criteria: 

(a)  Global  Technical  Proficiency  (TP) ; 

(b)  Global  Interpersonal  Proficiency  (IP) ; 

(c)  Global  Time-to-Proficiency  (TTP) ; 

(d)  Decomposed  Time-to-Proficiency  (DTTP) ,  and; 

(e)  Decomposed  Performance  (DPERF) . 
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Retraining  predictors: 

(a)  Cross-job  Retraining  Time  Estimates  (XJRTEs) ; 

(b)  Retraining  Climate  -  Situational  Constraints  (SITCON) ; 

(c)  Retraining  Climate  -  Supervisor  Support  (SUPSUP) ;  and 

(d)  Retraining  Climate  -  Organizational  Support  (ORGSUP) . 

Two  additional  predictors  of  retraining  success  were 
Retrainee  Armed  Forces  Qualification  Test  (AFQT)  scores  and  AFS- 
level  OLDs.  AFQT  scores  (a)  are  formed  as . composites  of  the 
ASVAB  Word  Knowledge,  Paragraph  Comprehension,  Arithmetic 
Reasoning,  and  General  Science  subtests  (see,  e.g.,  Earles  &  Ree, 
1992) ,  (b)  are  regarded  as  strong  indicators  of  general  cognitive 

ability  (Department  of  Defense,  1984),  and  (c)  were  obtained  from 
MEPS  data.  AFS- level  benchmarked  OLDs  were  obtained  from  AL. 


RESULTS 

Convergent  Validation  of  XJRTEs 

Recall  that  the  goal  of  the  convergent  validation  of  XJRTEs 
was  to  assess  relationships  of  XJRTEs  with  other  AFS-level 
predictors  of  retraining  ease.  Table  6  shows  correlations 
between  the  three  sets  of  XJRTEs  (based  on  analysis  of 
Nonretrainee  versus  Retrainee,  Supervisor,  and  Peer  TSQ  mean  MTP 
profiles)  and  (a)  the  OLDS  of  the  » Into-AFSCs"  represented  in  the 
TSQ  data  set,  (b)  OLDS  of  the  "FROM-AFSCs, "  and  (c)  the  MAGE-F/T 
variable  indicating  whether  retraining  was  within  (=0)  or  across 


Table  6 

Convergent  Validation  -  Descriptive  Statistics  and.  Correlations  with. 
Cross-Job  Retraining  Time  Estimates 


Correlation  with  XJRTEs 


Variable 

Mean 

SD 

Retrainee 

Supervisor 

Peer 

Into-AFS 

OLD 

101.10 

18.51 

. 267** 

.152** 

.164** 

From- AFS 

OLD 

104.07 

20.13 

.087** 

.126** 

.107** 

MAGE-F/T 

.  75 

.43 

. 068** 

.062** 

.058** 

Note.  N  =  2046  for  all  correlations. 

**p  <  .01 
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Thus  correlations  in  Table  6  are  at 


different  (=1)  aptitude  areas, 
the  AFS- level  of  analysis. 

As  predicted,  XJRTEs  correlated  positively  with  OLDs  of  the 
" Into-AFSs"  (top  row  of  Table  6)  supporting  the  idea  that  attaining 
proficiency  in  a  new  assignment  should  take  longer  in  new  jobs  that 
are  generally  more  difficult  to  learn.  Unexpectedly,  XJRTEs  also 
correlated  positively  with  OLDs  of  the  "From-AFSs. "  We  expected 
this  relationship  to  be  negative  under  the  rationale  that  it  would 
be  easier  to  retrain  from  AFSs  that  are  generally  more  (as  compared 
to  less)  difficult  to  learn.  On  the  other  hand,  the  positive 
relationship  may  reflect  the  nongeneralizability  of  specific 
technical  task  areas  that  are  associated  with  AFSs  having  , higher 
OLDs . 


Finally,  although  the  correlations  are  small,  the  positive 
correlations  between  MAGE-F/T  and  the  XJRTEs  (bottom  row  of  Table 
6)  indicate  a  tendency  for  XJRTEs  to  be  higher  for  From-  and  Into- 
AFS  combinations  that  have  different,  versus  the  same,  aptitude  . 
requirement  areas.  This  was  as  predicted.  Another  way  of  looking 
at  this  finding  is  in  terms  of  the  From-AFS  MAGE  x  Into-AFS  MAGE 
(M=i,  a=2 ,  G=3 ,  E=4)  interaction  with  XJRTEs  as  the  dependent 
variable.  We  analyzed  this  interaction  by  conducting  three  2 -way 
analyses  of  variance  (ANOVAs)  each  with  From-AFS  MAGE  and  Into-AFS 
MAGE  areas  as  the  main  effects  factors  and  the  three  XJRTEs  (i.e., 
based  on  Retrainee,  Supervisor,  and  Peer  MTP  ratings)  as  the 
respective  dependent  measures.  Table  7  shows  these  results. 

The  first  two  rows  in  Table  7  show  that  there  are  significant 


Table  7 

Convergent  Validation  -  Analysis  of  Variance  Results  for  Cross-Job 
Retraining  Time  Estimates 


Source 

df 

F- Values 

Retrainee 

Supervisor 

Peer 

1. 

From-AFS  MAGE 

(F) 

3 

18.00** 

23.46** 

18.13** 

2. 

Into-AFS  MAGE 

(T) 

3 

87.01** 

29.49** 

20.13** 

3. 

F  x  T 

9 

3.62** 

4.88** 

3.39** 

4  . 

Residual 

2030 

— 

— 

—  —  — 

**£  <  .01 
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differences  in  XJRTEs  both  as  a  function  of  the  From-AFS's  MAGE 
area  (i.e.,  it  is  more  difficult  to  retrain  FROM  some  AFS  MAGE 
areas  than  others)  and  the  To-AFS's  MAGE  area  (i.e.,  it  is  more 
difficult  to  retrain  INTO  some  AFS  MAGE  areas  than  others) .  The 
third  row  in  Table  7  confirms  the  interaction  effect  for  all 
three  sets  of  XJRTEs.  Table  8  shows  the  cell  means  (and  cell  ns) 
for  XJRTEs  based  on  Supervisor  ratings.  To  facilitate 
interpretation  of  the  interaction,  these  cell  means  are  plotted 
in  Figure  2 . 

Note  that  the  "G"  curve  in  Figure  2  is  generally  higher  than 
the  others,  indicating  that  retraining  is  predicted  to  be  more 
difficult  from  AFSs  having  General  aptitude  requirements  as 
compared  to  AFSs  with  other  MAGE  area  assignments..  Also,  the 
mean  heights  of  the  curves  over  the  "M"  and  "E"  points  on  the 
abscissa  of  the  plot  indicate  that  retraining  is  predicted  to  be 
generally  more  difficult  into  Mechanical  and  Electronic  than  into 
Administrative  and  General  AFSs.  These  patterns  illustrate  the 
main  effects  found  in  Table  7:  retraining  is  generally  more 
difficult  from  "G"  jobs  and  into  "M"  and  "E"  jobs. 

The  predicted  interaction  was  one  in  which  the  lowest  value 
above  each  point  on  the  abscissa  corresponded  to  the  curve  for. 
the  same  aptitude  area,  indicating  that  it  is  easiest  to  retrain 
into  an  AFS  in  a  particular  aptitude  area  from  another  AFS  in  the 
same  aptitude  area  than  from  an  AFS  in  another  aptitude  area. 

This  was  confirmed  for  the  Administrative  and  Electronic  From-/To- 
AFS  combinations,  but  not  for  the  Mechanical  and  General  aptitude 
areas.  That  is,  as  predicted,  the  lowest  points  above  the  "A"  and 
"E"  points  on  the  abscissa  were  for  retraining  from  "A'.'  jobs  and 
"E"  jobs,  respectively.  Counter  to  predictions,  retraining  was 
found  to  be  easiest  into  "M"  jobs  from  "E"  jobs,  and  into  "G"  jobs 
from  "A"  jobs. 

Finally,  we  predicted  that  the  positive  XJRTE  -  OLD 
relationship  would  be  stronger  for  retraining  movements  across, 
rather  than  within,  aptitude  areas.  We  tested  this  hypothesis 
using  moderated  regression  (Cohen  &  Cohen,  1975)  by  first 
regressing  XJRTEs  on  the  Into-Job  OLDs  (0)  and  the  MAGE-F/T  (M) 
variables  and,  in  a  second  step,  the  MxO  cross-product  term.  In 
addition  to  showing  the  standardized  regression  weights  for  the 
full  equations  (i.e.,  the  equations  including  the  cross-product 
term) ,  Table  9  shows  that  the  inclusion  of  the  cross-product  term 
resulted  in  a  significant  increase  in  R2  for  Retrainee-based  and 
Supervisor-based  XJRTEs  (but  not  for  Peer  XJRTEs) .  We  analyzed  the 
form  of  the  significant  interactions  by  calculating  predicted 
values  for  XJRTEs  (see  Jaccard,  Turrisi,  &  Wan,  1990)  at  the  lowest 
and  highest  OLD  values  observed  in  the  retrainee -populated  AFSs 
included  here  (i.e.,  67  and  130,  see  Table  2).  For  illustrative 
purposes,  the  interaction  effect  calculated  on  the  basis  of 
Supervisor-based  XJRTEs  is  shown  in  Figure  3 . 
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Mechanical 


6.30 

(94) 


6.46 

(160) 


6.27 

(352) 


7.21 

(128) 


Administrative 

General 

Electronic 


7.56 

4.87 

(66) 

(102) 

8.36 

6.60 

(78) 

(130) 

5.73 

6.04 

(30) 

(50) 

4.93 

6.98 

(242) 

(88) 

6.36 

8.38 

(275) 

(104) 

5.61 

6.00 

(110) 

(37) 

Table  9 


Convergent  Validation  -  Moderated  Regression  Results 

Standardized  Regression  Weights 


Source 


Retrainee 

Supervisor 

Peer 

1. 

MAGE-F/T  (M) 

- .301* 

- .305* 

-.171 

2. 

TO-AFS  OLD  (0) 

.222** 

.106** 

.135** 

3. 

M  X  0 

.364** 

.369** 

.227 

R2  change 

.004** 

.004** 

.002 

Figure  3  shows  that,  as  predicted,  a  positive  relationship 
was  supported  between  XJRTEs  and  OLDs  for  retraining  across 
different  MAGE  aptitude  areas,  whereas  the  relationship  was 
negligible  for  retraining  across  different  AFSs  within  the  same 
aptitude  area. 

Summary.  Results  of  the  convergent  validation  indicated 
that,  as  predicted,  XJRTEs  correlated  with  two  other  theoretical 
predictors  of  actual  cross- job  retraining  ease:  (a)  the  general 

occupational  learning  difficulty  of  the  new  job  and  (b)  movement 
into  a  new  job  with  different  aptitude  requirements  than  the  old 
job.  Additionally,  the  idea  that  the  general  learning  difficulty 
of  the  new  job  is  more  important  in  determining  retraining  ease 
to  the  extent  that  the  new  job  also  entails  different  aptitude 
requirements  than  did  the  old  job  was  also  supported. 


Postdictive  Validation  of  XJRTEs 

The  purpose  of  this  part  of  the  validation  effort  was  to 
link  XJRTEs  calculated  on  the  basis  of  data  collected  in  1994  to 
criteria  related  to  retraining  ease  collected  in  the  late  1970s 
as  part  of  the  Airman  Retraining  Program  Study  (ARPS) .  Recall 
that  we  tracked  the  identity  of  AFSCs  very  closely  across  several 
revisions  of  the  enlisted  USAF  structure  to  ensure  that  1994 
XJRTEs  referred  to  substantially  the  same  AFSs  we  retained  from 
the  ARPS  data  set  for  validation.  Table  10  shows  the  number  of 
research  participants  in  each  of  the  AFSs  we  were  able  to  include 
from  the  ARPS  data  set  for  postdictive  validation.  Entries  in 
the  columns  labelled  "From-"  and  "Into-AFS"  show  the  number  of 
ARPS  respondents  who  indicated  they  had  retrained  from  and  into 
these  AFSs,  respectively.  Thus  these  data  show  again  that  some 
AFSs  are  much  more  often  "sources"  for  retraining  (e.g.,  AFSCs 
452X4,  462X0,  and  811X0)  while  others  are  more  often  "targets" 
for  retraining  (e.g.,  AFSCs  242X0,  251X0,  and  661X0) . 

Table  11  shows  correlations  between  XJRTEs  (calculated  from 
1994  TSQ  data)  and  ARPS  criterion  measures  (obtained  from  ARPS 
retrainees  and  their  supervisors) .  These  results  provide  weak 
support  for  the  hypothesized  relationship  between  XJRTEs  and  ARPS 
retraining  and  performance  criteria.  All  three  XJRTEs  correlated 
significantly  (though  not  strongly)  with  ARPS  Retrainee  self- 
ratings  of  the  extent  to  which  skills  learned  on  the  previous  job 
were  utilized  in  the  new  job.  Only  the  Retrainee -based  XJRTE 
correlated  significantly  with  a  similar  Supervisory  rating  of 
retrainee  previous  skill  utilization.  No  correlations  between 
XJRTEs  and  performance  criteria  were  significantly  different  from 
zero.  However,  as  Lance,  Mayfield,  and  Gould  (1993)  pointed  out, 
and  as  we  will  reinforce  later  in  this  report,  XJRTEs  are  not 
intended  to  predict  performance  in  the  new  assignment  as  much  as 
they  are  intended  to  predict  time  to  achieve  proficiency  in  the 
new  assignment .  Thus  findings  in  Table  11  indicating  that  XJRTEs 
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correlate  significantly  only  with  criteria  measuring  the  extent 
to  which  the  new  job  draws  upon  skills  that  were  learned 
previously  in  the  old  job  are  not  unexpected. 


Table  10 

Postdictive  Validation  -  AFSs  Surveyed  and  Number  of  Research 
Participants 


AFSCs 

Title 

ARPS  Retrainees 

From-AFS  Into-AFS 

113X0 

Flight  Engineer 

14 

75 

114X0 

Aircraft  Loadmaster 

9 

84 

201X0 

Intelligence  Operations  Specialist 

20 

0 

207X1 

Morse  Systems  Operator 

0 

2 

242X0 

Disaster  Preparedness 

2 

109 

2  51X0 

Weather  Specialist 

8 

113 

272X0 

Air  Traffic  Control  Operator 

80 

84 

277X0 

Space  Systems  Operations  Specialist 

0 

2 

304X4 

Ground  Radio  Communications  Specialist 

36 

21 

3  05X4 

Elec  Computer  &  Switching  Systems  Spec. 

20 

56 

306X6 

Secure  Communications  Systems  Specialist 

32 

41 

361X1 

Communication  Cable  Systems  Specialist 

3 

13 

392X0 

Maintenance  Scheduling  Specialist 

3 

99 

452X4 

Tactical  Aircraft  Maintenance  Specialist 

316 

29 

454X0 

Aerospace  Propulsion  Specialist 

141 

54 

455X1 

Avionics  Control  Specialist 

71 

16 

455X2 

Avionics  Communications  Specialist 

19 

5 

457X0 

Strategic  Aircraft  Maintenance  Specialist 

11 

0 

462X0 

Aircraft  Armament  Systems  Specialist 

113 

10 

472X4 

Vehicle  Maintenance  Specialist 

0 

16 

496X0 

Comm- Computer  Sys  Ping  &  Prg  Specialist 

1 

22 

645X0 

Inventory  Management  Specialist 

133 

51 

645X1 

Materiel  Storage  &  Distribution  Specialist 

125 

24 

645X2 

Supply  Systems  Analysis  Specialist 

4 

73 

661X0 

Logistics  Plans  Specialist 

0 

109 

672X1 

Financial  Management  Specialist 

8 

38 

672X2 

Financial  Services  Specialist 

19 

26 

674X0 

Financial  Analysis  Specialist 

9 

63 

702X0 

Administration  Specialist 

390 

212 

732X0 

Personnel  Specialist 

50 

158 

734X0 

Social  Actions  Specialist 

16 

115 

751X1 

Training  Systems  Specialist 

1 

1 

811X0 

Security  Specialist 

207 

39 

811X2 

Law  Enforcement  Specialist 

6  2 

6  2 

881X0 

Paralegal  Specialist 

3 

67 

906X0 

Medical  Administrative  Specialist 

14 

51 

Total 

1940 

1940 

39 


Table  11 


Postdictive  Validation  -  Correlations  Between  XJRTEs  and  ARPS  Criteria 


Retraining  Time  Estimates 


ARPS  Criteria 

Re trainee 

Supervisor 

Peer 

Retrainee  Ratings: 

Previous  Skill  Utilization 

-  .183** 

-.113** 

-.098** 

Current  Skill/Ability 

-.002 

.015 

-.025 

\ 

Supervisor  Ratinqs : 

Previous  Skill  Utilization 

-  .086** 

-.049 

.003 

Current  Skill/Ability 

-.004 

.030 

.014 

Performance  Effectiveness 

-.011 

-.020 

.012 

Note.  N  =  1940  for  all  correlations. 

**E  <  .01 


Construct  and  Criterion-Related  Validation  of  XJRTEs 

This  section  presents  the  core  findings  from  this  study. 
Results  from  the  convergent  and  postdictive  validation  efforts 
substantially  replicate  and  thereby  corroborate  earlier  findings 
(Lance,  Kavanagh,  &  Gould,  1993;  Lance,  Mayfield,  &  Gould,  1993) . 
However,  previous  studies  on  transferability  of  skills  and  the 
validation  of  cross-job  retraining  time  estimates  have  repeatedly 
identified  the  need  for  construct  validation  of  XJRTEs  and 
linkages  between  XJRTEs  and  measures  of  actual  retraining  success 
following  reassignment.  Results  presented  in  this  section 
address  these  research  needs . 

Recall  that  data  from  the  TSQ  were  used  to  operationalize 
several  retraining-related  constructs  (i.e.,  predictors  of 
retraining  success  as  well  as  retraining  criteria)  each  from 
three  different  measurement  sources  (i.e.,  Retrainee,  Supervisor, 
and  Peer  ratings) .  As  such,  the  triangulation  data  collection 
rat egy  effected  here  permits  an  application  of  the  multitrait- 
multimethod  (MTMM,  Campbell  &  Fiske,  1959)  approach  to  the 
analysis  of  multiple  retrainincp'^^l^t-^^  constructs,  each  assessed, 
by  multiple  measurement  methods/sources.  This  permits  the 
analysis  of  relationships  among  retraining  latent  variables, 
while  simultaneously  controlling  for  measurement  source  effects 
in  observed  measures.  Figure  4  shows  the  general  MTMM  structure 
employed  here . 
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Figure  4 .  MTMM  Approach  to  Analysis  of  Relationships  Between  Retraining- 
Related  Constructs  and  Method  Factors 


Boxes  in  (the  center  of)  Figure  4  represent  observed 
measures.  Here,  the  prefixes  S,  R,  and  P,  refer  to  the 
measurement  source  (Supervisor,  Retrainee,  and  Peer, 
respectively) ,  and  the  remainder  of  the  acronym  refers  to  the 
construct  being  assessed,  that  is: 

TP  -  Global  Technical  Proficiency 

IP  -  Global  Interpersonal  Proficiency 

TTP  -  Global  Time -to- Proficiency 

DTTP  -  Decomposed  Time-to-Prof iciency 

RTE  -  Retraining  Time  Estimate  (i.e.,  XJRTEs) 

CLIM  -  Retraining  Climate  Variables 

\ 

Thus  each  box  represents  a  "trait -method  unit"  or  a  particular 
construct  as  operationalized  by  a  particular  measurement 
method/ source .  For  simplicity  of  presentation,  DPERF  - 
Decomposed  Performance  and  the  specific  Retraining  Climate 
measures  (i.e.,  Situational  Constraints,  Supervisory  Support,  and 
Organizational  Support)  are  not  shown  in  Figure  4,  but  should  be 
included  implicitly.  Also,  OLD  represents  the  Into-AFS's 
occupational  learning  difficulty,  and  AFQT  represents  the 
Retrainee's  Armed  Forces  Qualification  Test  score  composite,  a 
strong  indicator  of  general  cognitive  ability  (g) . 

The  circles  (and  ellipses)  at  the  top  of  Figure  4  represent 
latent  retraining-related  constructs  (common  factors) ,  whereas 
those  at  the  bottom  of  the  Figure  refer  to  common  factors 
representing  measurement  source  (method)  effects.  Curved, 
double-headed  arrows  represent  correlations  among  the  retraining 
constructs  and  among  the  measurement  source  latent  variables . 
Finally,  straight,  single-headed  arrows  indicate  inferred  causal 
effects  from  the  latent  variables  to  the  observed  measures.  Thus 
each  observed  measure  (e.g.,  STP)  is  presumed  to  reflect  variance 
relating  to  the  latent  retraining  construct  it  was  intended  to 
measure  (e.g.,  Global  Technical  Proficiency  -  TP)  as  well  as 
variance  attributable  to  the  particular  measurement  source  (e.g.. 
Supervisor  rating  -  S)  by  which  the  construct  was  _ 

operationalized.  Not  shown  for  the  sake  of  clarity,  is  a  third 
source  of  variance  attributable  to  nonsystematic  measurement 
error. 

A  number  of  approaches  to  the  analysis  of  MTMM  data  have 
been  proposed  since  Campbell  and  Fiske's  (1959)  introduction  of 
the  MTMM  matrix.  Although  debate  continues  over  the  most 
appropriate  parameterization,  confirmatory  factor  analysis  (CFA) 
is  still  widely  accepted  as  the  method  of  choice  for  the  analysis 
of  MTMM  data.  This  is  the  approach  we  chose  here  to  estimate 
relationships  between  the  observed  measures  and  the  retraining- 
related  and  measurement  source  latent  variables,  and  among  the 
latent  variables  themselves .  The  advantages  of  this  approach 
over  more  traditional  regression-related  approaches  include  (a) 
disattenuation  in  correlation  and  regression  parameter  estimates 
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for  measurement  error,  (b)  statistical  estimation  of  convergent 
and  discriminant  validity  of  observed  measures,  (c)  analysis  of 
relationships  among  constructs  rather  than  particular . 
realizations  (operationalizations)  of  them,  and  (d)  disentangling 
of  effects  attributable  to  constructs  of  interest  from  those  due 
to  measurement  sources/methods. 

We  used  LISREL-8  for  PC  (J&reskog  &  S&rbom,  1993)  to 
estimate  the  MTMM  "measurement  model"  implied  by  Figured.  The 
complete  covariance  matrix  included  covariances  among  nine 
constructs  (Global  Technical  Proficiency,  Global  Interpersonal 
Proficiency,  Global  Time-to-Prof iciency,  Decomposed  Time-to- 
Proficiency,  Decomposed  Performance,  XJRTEs,  and  the  three 
Retraining  Climate  variables:  Situational  Constraints, 

Supervisory  Support,  and  Organizational  Support)  each  as  measured 
by  three  measurement  methods  (i.e.,  Supervisor,  Retrainee,  and 
Peer  ratings) ,  plus  two  additional  variables  corresponding  to  the 
"Into-AFS's"  OLD,  and  the  retrainee's  AFQT  score.  Thus  the _ input 
matrix  was  a  29  x  29  covariance  matrix.  Specifically,  we  fit  an 
adaptation  of  the  additive  CFA  Model  3C  described  by  Widaman 
(1985)  in  which  we  specified  eleven  "trait"  factors  corresponding 
to  the  nine  retraining-related  constructs  plus  AFQT,  and  Into-AFS 
OLD  factors,  and  three  "method"  factors  corresponding  to  the 
three  rating  sources. 

The  LISREL  program  reached  a  convergent  maximum  likelihood 
(ML)  solution  for  the  measurement  model  in  64  iterations  with 
Chi-square (296)  =  772.43  (pc. 01).  Although  the  model  could  be 

rejected  statistically  (as  is  usually  the  case  with  large  models 
and  large  samples) ,  alternative  overall  goodness-of -f it  indices, 
including  Jbreskog  and  Sftrbom's  Goodness-of -Fit  Index  (GFI=.91), 
Bentler's  (1990)  Comparative  Fit  Index  (CFI=.89),  the 
Tucker-Lewis  index  (TLI=.84),  the  standardized  root  mean  squared 
residual  (RMSR=.049),  Browne  and  Cudeck ' s  (1993)  root  mean 
squared  error  of  approximation  (RMSEA=.054)  and  the  associated 
probability  of  close  fit  (p-close  =  .075)  indicated  reasonably 
acceptable  model  fit. 

ML  estimates  for  measurement  model  factor  loadings  are  shown 
in  Table  12.  Variables'  loadings  on  "trait"  factors  (first  nine 
columns)  and  "method"  factors  (last  three  columns)  can  be 
interpreted  as  partial  evidence  for  convergent  validity  and  the 
presence  of  method  effects,  respectively.  The  first  five  columns 
of  Table  12  show  measured  variables '  loadings  on  latent  Global 
Technical  Proficiency  (TP) ,  Global  Interpersonal  Proficiency  (IP) , 
Global  Time  To  Proficiency  (TTP) ,  Decomposed  Time  To  Proficiency 
(DTTP) ,  and  Decomposed  Performance  (DPERF)  criterion  variables. 
Statistically  significant  factor  loadings  indicate  significant 
convergent  validity  for  all  measures.  Note  that  loadings  for 
Retrainee  measures  have  the  lowest  factor  loadings  (convergent 
validities)  in  each  case.  This  is  consistent  with  previous 
findings  on  convergence  among  alternative  rating  sources 
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Fixed  parameter 


(e.g.,  Harris  &  Schaubroeck,  1988)  which  has  shown  that  peers; 
and  supervisors'  performance  ratings  converge  more  strongly  with 
each  other  than  either  does  with  incumbent  self-ratings.  The 
sixth  column  shows  convergence  among  XJRTEs  in  terms  of  their 
loadings  on  the  latent  Retraining  Time  Estimate  (XJRTE)  factor. 
These  are  among  the  highest  in  the  factor  pattern  matrix, 
indicating  strong  convergence  among  sources  in  estimating  cross¬ 
job  retraining  times.  The  next  three  columns  show . convergent 
validity  evidence  in  measuring  climate  for  retraining  in  terms  of 
climate  measures'  factor  loadings  on  the  three  Retaining  Climate 
latent  variables.  Again,  factor  loadings  were  statistically 
significant  in  all  cases.  Interestingly,  factor  loadings  swere 
highest  for  Supervisor  climate  variables,  suggesting  that  they 
have  a  broader  perspective  on  retraining  climate  than  do  either 
the  retrainees  or  their  peers. 

The  last  three  columns  show  measures '  loadings  on  rating 
source  factors.  Most  of  these  are  statistically  significant,  and 
can  be  interpreted  as  indicating  that  most  measures  reflect 
significant  "method"  effects.  Alternately,  these  effects  can  be 
interpreted  as  unique,  source-specific  perspectives  on  the 
constructs  being  rated  (Lance,  Teachout,  &  Donnelly,  1992) . 
Regardless  of  their  interpretation,  the  XJRTEs'  very  low  loadings 
on  these  factors  indicate  their  freedom  from  influence  of 
method/source  effects. 

In  general,  results  in  Table  12  indicate  (a)  significant 
convergent  (measurement)  validity  for  each  of  the  measures  in 
representing  the  underlying  retraining-related  latent  variables, 
(b)  significant,  but  uneven  measurement  method/source  effects  on 
measures,  and  (c)  the  high  fidelity  with  which  XJRTEs  were 
operationalized . 

Confirmatory  factor  intercorrelations  are  shown  in  Table  13 . 

First,  note  that  the  performance-related  factors  (i.e.,  GLBL 
TECH  PROF,  GLBL  INTP  PROF,  and  DCMP  PERF)  are  all  strongly  and 
significantly  intercorrelated  (rs  range  from  .77  to  .95).  This 
supports  the  idea  that  all  three  factors  reflect  slightly 
different  aspects  of  retrainee  general  performance  effectiveness 
in  the  new  assignment.  Second,  note  that  GLBL  TTP  and  DCMP  TTP 
are  significantly  intercorrelated  (i.e.,  r  =  .50,  p<.01).  This 
indicates  convergence  between  two  very  different  methods  of 
estimating  time  for  retrainees  to  achieve  proficiency  in  the  new 
assignment  (i.e.,  global  vs.  decomposed  estimation) .  Third,  note 
that  correlations  among  the  three  retraining  climate  factors  also 
are  relatively  high  and  statistically  significant  (rs  range  from 
|  .  56 |  to  | . 69 | )  .  This  supports  the  idea  that  although. 
Situational  Constraints,  Supervisor  Support,  and  Organizational 
Support  are  distinguishable  aspects  of  climate  for  retraining, 
they  might  also  relate  to  a  more  generalized  perception . of  the 
climate's  supportiveness  of  retraining  (e.g.,  a  Retraining 
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Climate  PCg,  see  James  &  James,  1989) .  Finally,  note  that 
correlations  among  the  three  source  factors  (i.e.,  SUPR,  RETR, 
and  PEER  METH  factors)  are  low  (although  two  are  statistically 
significant) .  These  lend  some  support  to  Lance  et  al. 1  s  (1992) 
hypothesis  that  the  source  factors  represent  unique  perspectives 
on  the  constructs  being  rated,  rather  than  "method  effects"  per 
se . 


The  most  important  correlations  in  Table  13  for  the  purposes 
of  criterion-related  validation  are  between  the  XJRTE  factor  and 
performance  and  retraining  criteria.  As  expected,  XJRTE 
correlated  significantly  with  both  criterion  factors  reflecting 
time  to  achieve  proficiency  in  the  new  assignment  (i.e.,  GLBL  TTP 
and  DCMP  TTP) .  It  is  important  to  remember  that  these 
correlations  are  among  the  retraining  time  estimate  and  time  to 
proficiency  latent  variables.  While  these  correlations  are  not 
strong  (r  =  .22  and  .25,  respectively),  they  do  support  the  idea 
that  job- level  similarities  and  differences  in  knowledge  and 
skill  requirements  are  important  in  determining  cross-job 
retraining  success.  These  correlations  also  provide  the  link,  at 
the  level  of  latent  variables,  between  estimates  of  cross-job 
retraining  ease  and  actual  retraining  ease  following  retrainee 
reassignment,  thus  answering  calls  from  previous  research  for 
empirical  linkages  between  retraining  ease  predictions  and 
measures  of  actual  retraining  ease. 

Correlations  between  XJRTE  and  performance  criteria  (i.e., 
GLBL  TECH  PROF,  GLBL  INTP  PROF,  and  DCMP  PERF)  were  not 
significantly  different  from  zero.  This  is  not  surprising,  as 
XJRTEs  were  intended  to  predict  the  ease  with  which  retrainees 
learn  to  perform  their  job  in  the  new  assignment,  not  necessarily 
their  level  of  performance  once  they  have  learned  the  new  job. 

Regarding  other  predictors  of  retraining  ease  criteria  (GLBL 
TTP  and  DCMP  TTP) ,  retraining  climate  did  not  consistently  relate 
to  retraining  ease,  contrary  to  predictions.  The  only 
statistically  significant  relationship  was  a  positive  correlation 
(r= .20)  between  Situational  Constraints  (SIT  CONST ) . and 
Decomposed  Time-to-Prof iciency  (DCMP  TTP).  As  predicted,  the 
general  learning  difficulty  of  the  new  job  related  to  retraining 
ease,  but  only  for  Decomposed  TTP  measures.  Unexpectedly,  airman 
aptitude  (RETR  AFQT)  was  unrelated  to  either  retraining  success 
factor,  though  it  was  related  to  overall  technical  proficiency. 

In  summary,  correlations  among  retraining-related . factors 
support  the  following  conclusions:  (a)  XJRTEs  are  valid 

predictors  of  actual  retraining  ease  following  reassignment,  (b) 
XJRTEs  are  valid  predictors  of  retraining  success,  but  not  of 
performance  in  the  new  job  per  se,  (c)  re trainee  aptitude  seems 
to  play  a  larger  role  in  determining  performance  rather  than  new 
job  learning,  and  (d)  climate  for  retraining  is  not  strongly 
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related  to  retraining  success. 

The  finding  that  climate  for  retraining  did  not  relate 
consistently  to  retraining  success  runs  counter  to  the  limited 
literature  on  this  topic  and  to  our  second  general  hypothesis. 
This  raises  the  question  of  whether  climate  for  retraining  might 
be  better  conceived  as  a  moderator  of  the  relationship  between 
old/new  job  similarities  and  differences  in  task  content  and 
3_ggufiiing  times  (as  captured  by  the  XJRTEs  developed  here)  and 
retraining  success . 

To  test  this  idea  (our  third  general  hypothesis) ,  we  \ 
conducted  a  series  of  analyses  among  the  latent  retraining 
factors  that  are  analogous  to  hierarchical  regression . analyses  at 
the  level  of  observed  measures.  Until  recently,  testing 
interactions  among  latent  variables  was  nearly  an  intractable 
problem.  However,  Jaccard  and  Wan  (1995)  showed  how  latent 
interaction  terms  can  be  implemented  in  the  LISREL-8  program  by 
including  (a)  cross-products  among  manifest  indicators  of  latent 
variables  which  have  inferred  interactive  effects  on  other  latent 
■yg iables ,  and  (b)  a  complex  series  of  inequality  constraints  on 
the  latent  interaction  terms'  variances,  and  cross-product  terms' 
factor  loadings  and  residual  variances. 

We  implemented  Jaccard  and  Wan's  (1995)  approach  here  to 

for  the  possible  moderating  effects  of  the  three  retraining 
climate  latent  variables  on  the  relationships  between  XJRTE  and 
the  two  TTP  latent  dependent  variables .  As  in  moderated 
regression  we  compared  the  fits  of  models  which  included  only 
linear  terms  linking  a  dependent  variable  (e.g.,  GLBL  TTP)  to 
XJRTE  and  a  retraining  climate  factor  (e.g.,  SIT  CONST)  to  a 
second  model  which  also  included  the  latent  interaction  term 
(e.g.,  XJRTE  *  SIT  CONST).  Differences  in  model  fit  were 

evaluated  by  the  change  in  %2  from  the  reduced  to  the  full  model. 
Table  14  shows  the  results. 

The  top  half  of  Table  14  shows  tests  of  interaction 
hypotheses  between  XJRTEs  and  each  of  the  retraining  climate 
latent  variables  on  the  Decomposed  TTP  latent  variable.  The 
bottom  half  shows  similar  results  for  the  Global  TTP  dependent 
variable.  Only  one  interaction  received  statistical  support, 
between  XJRTEs  and  Situational  Constraints  in  the  prediction  of 
Decomposed  TTP.  Thus  with  the  exception  of  this  one  interaction 
effect,  the  third  general  Hypothesis  was  not  supported. 

Nevertheless,  to  determine  the  pattern  of  the  one 
interaction  that  was  supported,  we  extended  Jaccard  et  al.'s 
(1990)  procedure  for  depicting  interaction  effects  among  measured 
variables  to  the  analysis  of  latent  interaction  effects. 

Standard  deviations  of  the  XJRTE  and  SIT  CONST  latent  variables 
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were  computed  from  the  square  roots  of  the  appropriate  diagonal 
elements  in  the  matrix  of  covariances  among  latent  variables 
provided  by  the  LISREL  program.  Predicted  values  of  Decomposed 
TTP  were  computed  from  substituting  high  and  low  values  (i.e.,  +2 
SDs)  of  XJRTE  and  SIT  CONST  into  the  unstandardized  full 
regression  equation  (i.e.,  that  included  latent  linear  and 
interaction  terms) .  The  resulting  plot  is  shown  in  Figure  5. 

Although  the  interaction  shown  in  Figure  5  was  the  only  one 
that  was  statistically  significant  out  of  the  six  tested,  its 
form  was  as  predicted.  Time-to-prof iciency  (i.e.,  difficulty  of 
retraining)  was  predicted  to  be  uniformly  high  under  constraining 
climates  (i.e.,  High  Situational  Constraints).  However,  in 
unconstrained  climates,  XJRTEs  were  strongly  and  positively 
related  to  retrainee  time-to-prof iciency .  Generally,  in  climates 
characterized  by  situational  constraints,  old/new  job 
similarities  and  differences  in  task  contents  and  learning  times 


Table  14 

Tests  of  XJRTE  x  Retraining  Climate  Interactions  in  Predicting  TTP 


Dependent  Variable:  Decomposed  TTP 

Linear  Only 

Linear  Plus 
Interactive 

Change 
in  Fit 

Climate  Variable: 

df 

fO 

df 

x2 

R2 

Ax2 

AR2 

S i t ua t iona 1  Con s  t  ra in t  s 

57 

182.78  .12 

56 

177.18 

.  18 

5.60* 

.06 

Supervisor  Support 

57 

219.59  .12 

56 

218.92 

.12 

.67 

.00 

Organizational  Support 

57 

266.10  .09 

56 

263.72 

.13 

2.38 

.04 

Dependent  Variable:  Global  TTP 

Linear  Only 

Linear  Plus 
Interactive 

Change 
in  Fit 

Climate  Variable: 

df 

X2  E2 

df 

x2 

R2 

Ax2 

AR2 

Situational  Constraints 

57 

172.36  .05 

56 

169.47 

.07 

2.89 

.02 

Supervisor  Support 

57 

197.78  .06 

56 

197.68 

.06 

.10 

.00 

Organizational  Support 

57 

251.14  .05 

56 

249.54 

.06 

1.60 

.01 

*  E  <  .01 
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Figure  5.  Situational  Constraint  (SITCON)  x  XJRTE 
Interaction  Effect  on  Decomposed  TTP 
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make  little  or  no  difference  in  the  ease  of  retraining  - 
retraining  is  difficult  for  all.  It  is  in  situations  in  which 
there  are  fewer  constraints  on  learning  and  performance  that  the 
relationship  between  old/new  job  differences  and  retraining  ease 
itself  is  less  restricted,  and  tends  to  be  strongly  positive. 


DISCUSSION 

Overall  Project  Objective 

The  overall  and  major  purpose  of  this  research  project  was 
to  provide  criterion-related  validity  for  the  methodology  for 
estimating  cross-iob  retraining  times  (XJRTEs) .  This  was ^ a 
critical  step  for  the  USAF  to  have  an  operational  capability  to 
develop  a  "transferability  of  skills  matrix"  of  retraining  time 
estimates  for  the  approximately  180  enlisted  AFSs.  The  empirical 
results  of  this  project  clearly  indicate  success  in  meeting  this 
major  objective  of  providing  criterion-related  validity  for  the 
use  of  this  methodology  in  estimating  XJRTEs.  Further,  the  use 
of  the  MTMM  design  (Campbell  &  Fiske,  1959)  and  related  analyses 
should  provide  added  confidence  in  the  results. 


Tests  of  Hypotheses  and  Completion  of  Task  Activities 

Consistent  with  the  goal  of  developing  a  methodology  to 
establish  a  "transferability  of  skills  matrix"  for  enlisted  AFSs, 
this  research  project  tested  three  general  hypotheses.  We 
discuss  the  results  for  each  hypothesis  in  this  section. 

In  terms  of  our  first  general  hypothesis,  the 
criterion-related  validity  results  obtained  in  the  MTMM  design 
(see  Figure  4)  provided  strong  evidence  that  XJRTEs  are  valid 
predictors  of  actual  retraining  ease  following  reassignment .  The 
significant  relationships  reported  in  Table  13  between  XJRTEs  and 
both  of  the  criterion  variables  measuring  time  to  achieve 
proficiency  in  the  new  assignment  support  hypothesis  1,  and 
indicate  the  methodology  used  to  estimate  and  calculate  cross-job 
retraining  times  can  be  used  operationally  with  confidence  in  the 
Air  Force . 

The  fact  that  the  XJRTEs  were  not  significantly  related  to 
the  job  performance  criteria  should  not  be  interpreted  as 
limiting  their  usefulness.  Most  of  the  sample  of  retrainees  had 
been  on  their  new  jobs  for  some  time  before  our  data  collection 
occurred.  Since  there  are  multiple  influences  on  job  performance 
after  the  initial  learning  period  on  a  new  job,  our  results  are 
not  surprising.  Perhaps  the  most  important  aspect  of  performing 
on  a  new  job  during  the  first  few  weeks  is  the  ease  and  speed  of 
becoming  proficient  in  the  job  tasks.  Certainly,  this  is  the 
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critical  aspect  of  the  reassignment  and  training  programs  within 
the  Air  Force.  Thus,  our  results  in  validly  predicting 
time-to-prof iciency  in  new  assignments  may  be  interpreted  as  the 
most  important  job  performance  criteria  to  be  used  in  the 
evaluation  of  enlisted  reassignment  program. 

The  second  general  hypothesis  of  this  research  project  was 
not  consistently  supported  as  can  be  seen  in  Table  13 .  It  may  be 
that  the  organizational  climate  variables  included  in  this 
research,  with  one  exception,  are  not  critical  to  ease  of 
movement  to  a  new  assignment  for  enlisted  personnel.  However, 
these  results  may  reflect  one  of  the  inescapable  dilemmas  of 
field  survey  research  --  the  timing  of  the  measurement  of  the 
variables.  Organizational  Support  for  Continuous  Improvement, 
Supervisor  Support  for  Transfer  of  Training,  and  Situational . 
Constraints  may  be  important  influences  on  ease  of  movement  into 
a  new  job  in  the  very  early  stages,  for  example,  within  the  first 
two  weeks  to  a  month;  but  not  as  important  after  that  point  in 
time.  In  an  ideal  longitudinal  design  that  would  allow  precise 
testing  of  our  hypotheses,  we  would  be  able  to  monitor  and 
measure  all  research  variables  continuously.  In  this  project, 
this  ideal  design  would  have  meant  continuous  measurement,  for 
example,  daily  or  weekly,  of  the  research  variables  from  the 
initial  reassignment  date  through  the  first  six  months  in  the  new 
job.  Obviously,  this  is  not  feasible  for  a  number  of  reasons, _ 
and  therefore,  we  must  be  satisfied  with  a  data  collection  design 
that  may  be  insensitive  to  timing  of  organizational  events  and 
introduces  some  error  into  the  data  collection  and  analyses. 

The  argument  in  the  previous  paragraph  is  important  because 
of  the  significant  relationship  found  between  Situational 
Constraints  and  the  Decomposed  Time-To-Prof iciency  criterion 
measure.  This  criterion  measure  was  based  on  a  summation  of  the 
ratings  of  time-to-prof iciency  on  each  of  the  relevant  task 
categories  (those  checked  as  "part  of  job")  ,  and  was  computed  for 
all  three  sources  --  Retraineees,  Peers,  and  Supervisors.  It  is 
likely  that  retrainees,  who  may  have  received  formal  training  in 
a  relatively  "constraint -free"  environment,  would  react . 
negatively  when  faced  with  a  real  job  environment  containing _ 
constraints  on  performance.  This  initial  shock  would  most  likely 
influence  early  job  performance  in  the  new  assignment,  and  thus, 
time-to-prof iciency  on  tasks.  The  possible  implications  of  this 
finding  on  formal  and  on-the-job  training  (OJT)  will  be  discussed 
in  a  later  section. 

Underscoring  this  finding  of  a  significant  relationship 
between  Situational  Constraints  and  Decomposed 

Time-To-Prof iciency  is  the  one  significant  interaction  result 
(see  Figure  5)  that  provides  some  support  for  the  third 
hypothesis.  Estimated  cross- job  retraining  time  was  found  to  be 
uniformly  high  in  unit  assignments  high  in  situational 
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constraints.  On  the  other  hand,  in  unit  assignments  with  a  low 
amount  of  situational  constraint,  XJRTEs  were  strongly  and 
positively  related  to  retrainee  time-to-prof iciency .  It  would 
appear  that  situational  constraints,  as  measured  in  this  project, 
are  affecting  the  learning  and  job  performance  of  recently 
reassigned  airmen  to  some  extent.  This  finding  suggests  that 
future  research  should  examine  how  these  situational  constraints 
influence  time-to-prof iciency  in  new  job  assignments,  and 
investigate  ways  to  modify  formal  and  on-the-job  training  to 
counteract  these  potential  negative  effects  or  enhance  the 
positive  effects  of  creating  unit  environments  relatively  free  of 
these  constraints.  \ 


Implications  for  the  Air  Force 
Cross-job  Retraining  Time  Estimates  Methodology 

The  major  payoff  from  this  research  is  a  XJRTE  methodology 
for  developing  a  "transferability  of  skills  matrix."  It  should 
now  be  possible  to  construct  a  matrix  for  a  set  of  "FROM"  and 
"TO"  enlisted  AFSs  containing  matrix  cell  values  representing 
ease  of  movement  (EOM)  out  of  and  into  the  targeted  AFSs.  The 
cells  of  the  transferability  of  skills  matrix  corresponding  to 
significant  numbers  of  transfers  (either  into  or  from)  based  on 
historical  data  are  of  most  interest.  However,  it  may  also  be 
possible  to  provide  estimate  EOM  values  for  important  AFSs  with 
low  historical  frequencies  of  transfers,  or  small  numbers  of 
airmen  populating  them. 

In  order  to  illustrate  the  use  of  the  XJRTEs  based  on  this 
methodology,  a  "FROM"  and  "TO"  matrix  of  values  for  eight  AFSs 
sampled  in  this  project  was  created  in  Table  15.  Interpreting 
this  matrix  may  be  aided  by  referring  to  Figure  1  and  our 
discussion  of  it  earlier.  In  Table  15,  we  show  a  matrix  of 
XJRTEs  based  on  profile  comparisons  between  Nonretrainee  and 
Retrainee  mean  MTP  profiles. 

Two  things  should  be  noted  about  this  matrix.  First,  the 
values  indicate  "relative  difficulty  of  retraining"  from  one  AFS 
to  another.  For  example  in  Table  15,  a  reassignment  from  AFSC 
113X0  to  AFSC  277X0  (4.47)  would  be  easier  than  a  reassignment 
from  AFSC  277X0  to  AFSC  113X0  (9.44).  In  general,  the  higher  the 
values  in  Table  15,  the  more  difficult  the  retraining 
reassignment.  Second,  as  can  be  seen  from  the  "XXX"  values  under 
the  811XX  column,  there  is  little,  if  any  transfer  into  this  AFS. 
This  is  consistent  with  the  reality  of  the  AFS  reassignment 
program . 

However,  there  are  cautions  one  must  observe  in  interpreting 
the  values  in  this  table.  First,  although  we  established 
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Table  15 


XJRTEs  Example  for  Eight  AFSs  Using  Nonretrainee  and  Retrainee  MTP  Profiles 


From 

AFS  : 

Into  -  AFSs 

113X0 

277X0 

452X4 

455X4 

457X0 

602XX 

732X0 

811XX 

113X0 

XX. XX 

4.47 

6.65 

12.09 

9.09 

8.86 

11.77 

XX. XX 

277X0 

9.44 

XX.  XX 

5.77 

13.78 

10.26 

7.07 

7.69 

XX.  XX 

452X4 

6.24 

2.48 

XX.  XX 

10.01 

4.89 

3.42 

8.11 

XX,  XX 

455X2 

7.74 

3.65 

6.51 

XX.  XX 

8.35 

5.57 

10.05 

XX. XX 

457X0 

8.99 

3.24 

4.75 

12.30 

XX.  XX 

5.22 

9.40 

XX. XX 

602XX 

8.05 

2.29 

5.20 

13.02 

6.67 

XX.  XX 

6.98 

XX.  XX 

732X0 

10.14 

3.08 

7.77 

17.25 

11.12 

6.77 

XX.  XX 

XX. XX 

811XX 

13.44 

6.67 

9.38 

18.98 

13.69 

9.83 

11.40 

XX.  XX 

Note.  XJRTE  values  are  relative  measures  of  months -to-prof iciency  (MTP)  for 
retraining  out  of  ("From")  and  into  the  indicated  AFSs. 


criterion-related  validity  for  the  XJRTE  methodology  in  this 
project,  the  validity  coefficient  was  only  .22.  That  is,  our 
validity  was  not  perfect.  Second,  when  one  considers  the 
potential  problems  introduced  by  sampling  errors,  it  would  be 
best  to  consider  the  values  in  Table  15  as  estimates  of  the 
retraining  difficulty  between  AFSs.  This  project  was  concerned 
with  validating  the  XJRTE  methodology,  not  producing  a 
"transferability  of  skills  matrix"  per  se.  Third,  the  values  in 
Table  15  represent,  at  best,  an  ordinal  ranking  of  ease  of 
movement  among  those  AFSs  sampled.  That  is,  one  could  not  assume 
that  it  would  be  twice  as  difficult  to  move  from  AFSC  277X0  to 
AFSC  113X0  (9.44)  than  vice-versa  from  AFSC  113X0  to  AFSC  277X0 
(4.47)  even  though  the  numbers  would  indicate  this.  The  numbers 
come  from  Months -to -Prof iciency  ratings  across  45  task  categories 
that  are,  in  all  likelihood,  different  for  different  AFSs.  Thus, 
comparisons  with  interval  or  ratio  measurement  are  inappropriate. 

With  these  cautions  in  mind,  what  can  be  done?  With  proper 
sampling,  a  transferability  of  skills  matrix  for  the _ 
approximately  180  AFSs  could  be  created.  Perhaps  this  data, 
collection  effort  could  be  combined  with  other  data  collections 
routinely  conducted  by  the  Air  Force  Occupational  Measurement 
Squadron .  However,  it  would  be  advisable  that  the  first  attempt 
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to  collect  these  data  and  establish  this  "transferability  of 
skills  matrix"  be  a  research  effort  to  ensure  that  the  necessary 
sampling  procedures  are  followed  and  that  the  reliability  of  the 
procedure  be  ascertained.  It  would  also  be  possible  to  create 
reassignment  software  to  handle  re-enlistment  assignments  of 
first- term  airmen.  Naturally,  this  would  require  data  collection 
using  the  TSQ  with  the  common  task  taxonomy  from  all  AFSs  to  be 
included  in  the  reassignment  software.  It  would  also  require  the 
calculation  of  XJRTEs  be  embedded  in  the  software.  This  software 
development  is  a  significant  application  effort,  however,  and 
importantly,  it  is  now  feasible  given  the  validity  evidence  for 
the  XJRTE  methodology  found  in  this  project  and  previous  validity 
research  efforts  already  cited. 

Other  research  and  operational  use  implications  of  the 
results  of  the  validation  of  XJRTES  are  discussed  in  more  detail 
in  later  sections  of  this  report. 


Organizational  or  Unit  Climate 

There  are  two  important  implications  for  the  USAF  on  the 
basis  of  the  results  we  found  for  the  organizational  climate 
variables.  The  first  one  has  been  alluded  to  earlier  in  this 
report;  that  is,  the  implications  for  formal  training  and  OJT  of 
the  significant  results  of  the  Situational  Constraints  variable. 
It  appears  that  situational  constraints  affect  time-to-  . 
proficiency  for  retrainees,  and  under  unit  conditions  with  little 
or  no  constraints,  XJRTEs  are  strongly  and  significantly  related 
to  retrainee  time-to-prof iciency.  Taken  together,  these  results 
indicate  that  situational  constraints  are  affecting  the 
ease -of -movement  of  retrainees  into  their  new  assignments.  We 
suspect  and  hypothesize  that  the  effects  of  situational 
constraints  on  ease -of -movement ,  particularly  in  terms  of 
time-to-prof iciency  on  relevant  task  performance,  are  most 
important  at  the  early  stage  of  the  new  assignment  --  within  the 
first  month. 

The  hypothesized  effect  of  these  situational  constraints  on 
time-to-prof iciency  early  in  the  reassignment  needs  further 
empirical  investigation.  However,  the  findings  of  this  project 
provide  guidance  to  the  training  community  in  regards  to  the 
design  of  formal  training  programs  as  well  as  OJT.  Evaluation  of 
training  programs  using  subsequent  job  performance  in  terms  of  a 
time-to-prof iciency  variable,  should  consider  the  units' 
situational  constraints  as  a  possible  moderating  variable. 
Situational  constraints  might  be  systematically  embedded  in 
training  programs  such  that  airmen  transferring  into  a  new  AFS 
would  have  some  experience  with  task  performance  in  less  than 
optimal  unit  conditions.  Further,  designers  of  OJT  programs 
should  be  cognizant  of  the  potential  influence  of  situational 
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factors  on  task  learning  and  performance  for  retrainees. 

The  second  implication  is  the  identification  of  three 
reliable  measures  of  unit  or  organizational  climate  from 
retrainees,  their  supervisors  and  their  peers.  There  are 
numerous  measures  of  organizational  culture  and  climate  available 
in  the  literature  (Schneider,  1990) ,  however,  to  our  knowledge, 
all  were  developed  using  a  single  data  source  for  the 
psychometric  analyses  to  establish  the  reliability  and  validity 
of  the  measures.  In  this  case,  we  have  used  three  sources  to 
identify  three  scales  with  items  in  common  to  measure  unit  or 
organizational  climate.  \ 

The  three  climate  scales  can  be  useful  in  future  research. 

They  have  immediate  applicability  for  projects  in  which  unit  or 
organizational  climate  is  included  as  a  research  variable,  such 
as  those  addressing  leadership,  team  or  group  effectiveness,  or 
any  organizational  change  effort.  The  situational  constraints 
measure  appears  especially  relevant  to  research  on  individual  or 
team  performance  measurement.  Finally,  as  indicated  above,  these 
measures  are  pertinent  to  transfer  of  training  research.  In 
short,  the  climate  scales  could  be  incorporated  in  any  research 
project  with  unit  climate  or  culture  as  a  critical  variable. 


Cross-level  Issues  and  Applications 

"Broadly  defined,  cross-level  inference  occurs  when 
relations  among  variables  at  one  level  are  inferred  from  analyses 
performed  at  a  different  level"  (Mossholder  &  Bedeian,  1983,  p. 
547) .  Empirically,  cross-level  linkages  are  made  between 
variables  operationalized  at  one  level  of  analysis  (e.g.,  the  job 
or  the  organization)  and  some  other  level  (e.g.,  the  individual) 
(see,  e.g.,  Lance,  Hedge,  &  Alley,  1989;  Rousseau,  1978;  1985; 
Schneider  &  Bowen,  1985;  Sutton  &  Rousseau,  1979;  Wallace,  1983) . 

XJRTEs  developed  here  (and  in  previous  research,  e.g., 

Lance,  Kavanagh,  &  Gould,  1993;  Lance,  Mayfield,  &  Gould,  1993; 
Lance,  Mayfield,  Gould,  &  Lynskey,  1991)  were  formed  at  the  AFS- 
level  of  analysis.  Recall  that  XJRTEs  were  basedon  mean  subject 
matter  expert  judgments  of  time  required  by  a  typical  airman  to 
become  proficient  on  task  areas  defined  by  the  taxonomy  shown  in 
Appendix  B.  Thus  XJRTEs  were  designed  to  index  the  average 
relative  difficulty  of  retraining  from  AFS*  to  AFSj  and  vice 
versa,  irrespective  of  individual -level  retrainee  characteristics 
(e.g.,  aptitude,  motivation  to  retrain,  etc.). 

Much  of  the  earlier  validation  research  on  XJRTEs  was  also 
conducted  at  the  AFS-level  of  analysis.  For  example,  in  their 
convergent  validation  study,  Lance,  Kavanagh,  and  Gould  (1993) 
found  that,  as  predicted,  XJRTEs  correlated  with  old-new  AFS 


56 


differences  in  general  occupational  learning  difficulties  and 
aptitude  requirements.  AFS-level  convergent  validity  evidence 
also  was  reported  earlier  in  this  report  in  our  replication  of 
Lance,  Kavanagh,  and  Gould's  (1993)  study.  Also,  Lance, 

Mayfield,  Gould,  and  Lynskey  (1991)  found  that  "decomposed"  job- 
level  XJRTEs  (such  as  those  reported  here)  exhibited  significant 
convergent  validity  with  "global"  XJRTEs  based  on  SMEs'  overall 
judgments  of  retraining  times  across  a  number  of  AFSs. 

Lance,  Mayfield,  and  Gould's  (1993)  postdictive  validation 
study  of  XJRTEs  represents  the  first  cross -level  study  on 
transferability  of  skills.  In  this  study  they  validated  AFS- 
level  XJRTEs  against  individual - level  criteria  related  to  the 
actual  ease  with  which  retrainees  had  achieved  proficiency  in  a 
new  assignment.  The  postdictive  validation  section  of  this 
report  substantially  replicated  these  findings.  Further,  the 
construct  and  criterion-related  validation  evidence  we  reported 
earlier  is  cross-level  in  the  sense  that  linkages  are  made 
between  AFS-level  XJRTEs  and  individual -level  retraining  success 
criteria.  However,  there  are  other  levels -of -analysis  issues, 
and  potential  applications  of  the  methodology  supporting  the 
estimation  of  cross-job  retraining  times  (XJRTs)  that  have  not 
yet  been  addressed. 

Theoretically,  there  are  potentially  multiple  levels  of 
analysis  to  which  this  methodology  might  be  applied.  These  range 
from  the  very  micro  level  of  discrete  task  steps,  to  tasks,  to 
task  modules,  to  job  dimensions,  to  organizational  positions,  to 
jobs,  to  job  families,  and  perhaps  entire  organizations.  We 
operated  at  the  level  of  the  AFS  in  estimating  XJRTs,  and  at  the 
level  of  the  AFS  and  position  (individual)  in  validating  XJRTEs. 
But  in  principle,  the  methodology  of  estimating  XJRTs  could  be 
extended  to  any  level  of  analysis  and  across  (at  least  adjacent) 
levels.  The  following  are  six  cross-level  extensions  that  we 
envision . 

Job  placement.  In  this  scenario,  AFS-level  months -to- 
proficiency  (MTP)  profiles  would  be  available  for  all,  or  a 
substantial  majority  of  AFSs  from,  for  example,  an  AFS-wide 
survey  using  an  adaptation  of  the  Nonretrainee  version  of  the 
TSQ.  Then  as  part  of  an  airman's  assessment  for  placement  into 
the  first  job  assignment,  the  airman  would  complete  a  similar 
inventory  of  the  airman's  present  skill  level  with  respect  to  the 
same  task  categories  that  comprise  the  job-level  MTP  profiles. 
Using  a  computational  algorithm  similar  to  the  one  we  employed  to 
estimate  XJRTs,  the  airman  could  be  presented  with  a  list  of 
possible  placement  assignments  ordered  by  predicted  ease  of  entry 
based  on  the  match  between  the  new  airman's  current  skill  mix  and 
the  eligible  AFSs'  skill  requirements.  Thus  this  would  represent 
a  situation  in  which  the  "From-Job"  is  actually  an  individual, 
whose  individual  skill  profile  is  assessed  against  (normative) 
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AFS-level  profiles  for  the  purpose  of  assessing  potential  person- 
job  fit.  Of  course,  the  same  methodology  would  be  applicable  to 
subsequent  job  assignments  (e.g.,  at  the  time  a  re-enlistment 
decision  is  made) ,  for  example,  in  supporting  cross-job 
retraining  decisions. 

Retraining  options.  The  levels-of -analysis  paradigm  is  the 
same  for  this  potential  application  as  it  is  for  the  job 
placement  scenario  -  assessment  of  individuals'  skill  profiles 
against  (normative)  job-level  skill  requirement  profiles. 

However,  the  application  is  in  the  retraining  arena,  and  is 
relevant  to  airmen  desiring  retraining,  airmen  affected  by  the 
elimination  of  their  AFS,  or  airmen  targeted  for  retraining  due 
to  personnel  overages.  Here,  inventories  of  individuals^  skills 
profiles  could  be  assessed  against  AFS-level  skills  requirements 
profiles  in  order  to  specify  retraining  options . based  on 
projections  of  the  relative  ease  with  which  individuals  might 
retrain  into  selected  retraining  AFSs.  Thus  this  application 
could  be  useful  to  individual  airmen  in  choosing  among  retraining 
options  and  to  the  USAF  in  allocating  retraining  decisions  to 
minimize  retraining  costs. 

AFS  consolidation.  XJRTEs  as  they  are  presently  implemented 
have  direct  implications  for  AFS  consolidation  -  AFSs  would  more 
readily  be  candidates  for  consolidation  to  the  extent  that 
retraining  across  AFSs  is  predicted  to  be  easier.  This  is  a 
levels-of -analysis  issue  in  the  sense  that  in  combining  AFSs  a 
higher  level  "new"  AFS  is  created.  Thus  the  methodology  of 
estimating  XJRTs  should  be  useful  in  making  "aggregation'1 
decisions,  or  decisions  to  consolidate  more  specific  entities 
(i.e.,  AFSs)  into  higher- level  entities  (i.e.,  AFS  families)  so 
that  retraining  for  performance  in  the  newly  aggregated  AFSs  is 
minimized . 

Manpower,  personnel  and  training  (MPT)  forecasting.  The 

XJRTE  methodology  described  and  validated  here  also  could  be 
applied  to  jobs  that  do  not  yet  exist.  For  example,  if  forecasts 
can  be  made  for  the  kinds  of  tasks  that  would  be  performed  on. 
proposed  weapons  systems,  estimates  of  the  learning  difficulties 
of  these  tasks  can  be  obtained,  and  the  configurations  of  tasks 
into  positions  and  AFSs  can  be  manipulated,  then  the  taxonomy  in 
Appendix  B  (or  some  version  of  it)  could  be  useful  in  (a) 
organizing  tasks  into  broader  task  categories,  (b)  assessing  the 
relative  difficulties  of  projected  AFSs,  (c)  assessing  personnel 
and  (re) training  implications  based  on  projected  AFS  structures, 
and  (d)  possibly  making  weapons  systems  design  modifications 
based  on  personnel  and  training  implications. 

Synthetic  validation.  Finally,  the  methodology  for 
estimating  XJRTs  is  adaptable  to  support  synthetic  validation. 

The  concept  of  synthetic  validation  (or  job  component  validity; 
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Balma,  1959;  Guion,  1965;  Hollenbeck  &  Whitener,  1988;  Lawshe, 
1952;  Mossholder  &  Arvey,  1984)  is  not  popular  today  with  current 
emphasis  on  general  cognitive  ability  (e.g.,  Herrnstein  &  Murray, 
1994)  and  broad  personality  factors  (e.g.,  Hogan,  1991). 
Nevertheless,  there  may  be  situations,  for  example  where  sample 
sizes  are  insufficient  or  when  the  jobs  do  not  yet  exist, 
which  preclude  more  traditional  validation  studies.  Using  a 
combination  of  the  task  taxonomic  approach  described  here  (e.g.. 
Appendix  B)  and  validation  strategies  described  by  Guion  (1965) , 
Hollenbeck  and  Whitener  (1988)  and  Mossholder  and  Arvey  (1984) , 
synthetic  validation  would  be  supported  by  analyzing  (projected) 
AFSs  into  components,  estimating  validities  for  job  components, 
and  aggregating  component  validities  into  overall  AFS-level  test 
validity  estimates. 


Implications  for  the  Scientific  Community 

From  a  scientific  viewpoint,  this  research  is  of 
considerable  importance  for  two  major  reasons.  First,  as  noted 
in  the  introduction  of  this  report,  although  transferability  of 
skills  has  received  some  attention  in  the  literature  (Byrne, 

1975;  Downs,  1985;  Fine,  1957a,  1957b;  Fossum  et  al . ,  1986; 

Magnum  &  Ball,  1987;  Rumberger,  1981),  there  has  been  no 
empirical  evidence  supporting  a  method  to  estimate 
transferability  across  -jobs  in  different  job  families.  This 
research  has  produced  a  scientifically  reliable  and  valid 
methodology  to  estimate  relative  retraining  times  across  jobs  in 
differing  career  fields.  This  is  the  first  time  this  has  been 
accomplished,  and  the  implications  for  research  in  the  fields  of 
selection,  job  transfer,  career  planning,  company  mergers,  and 
organizational  downsizing  are  tremendous.  The  complete 
specification  of  all  of  the  research  and  organizational 
implications  goes  beyond  the  boundaries  of  this  project  and  will 
not  be  discussed. 

The  second  major  importance  of  this  research  is  the  use  of 
the  multitrait -multimethod  (MTMM)  design  (Campbell  &  Fiske,  1959) 
for  predictors,  climate  measures,  and  criteria  in  this  validity 
study.  Close  reading  of  the  original  description  of  the  MTMM 
approach  (Campbell  &  Fiske,  1959)  reveals  that,  from  a  philosophy 
of  science  perspective,  the  authors  were  referring  to  the  use  of 
latent  variables  based  on  multiple  sources  as  the  "correct  and 
true"  approach  to  measurement  and  research  in  the  behavioral 
sciences .  The  primary  use  of  the  MTMM  approach  has  been  to 
establish  the  construct  validity  of  measures  of  single  constructs 
or  variables,  and  the  literature  is  filled  with  research  studies 
(e.g.,  see  Kavanagh  et  al.,  1971)  exemplifying  this  approach. 
However,  we  are  unaware  of  any  studies  that  have  used  the  MTMM 
design  to  establish  the  construct  validity  of  (latent)  variables, 
and  then,  examined  the  relationships  among  these  variables. 
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This  criterion- related  validity  study,  which  includes  a 
measure  of  organizational  climate  as  a  potential  (latent) 
moderator  variable,  is  the  only  one  of  which  we  are  aware . in  the 
scientific  literature  to  date.  Clearly,  this  effort  provides  a 
demonstration  that  the  "true"  approach  to  MTMM,  as  envisioned  by 
Campbell  and  Fiske  (1959),  can  be  done.  This  is  both  a  challenge 
and  opportunity  for  the  scientific  community,  and  we  hope  our 
effort  leads  to  better  use  of  the  MTMM  approach  in  the  future. 


Conclusions  \ 

The  most  important  conclusion  of  this  research  is  that  the 
methodology  for  estimating  XJRT  has  demonstrated  predictive 
validity  for  time-to-prof iciency  of  retrainees  and,  combined  with 
the  other  validity  studies  completed,  provides  a  powerful  tool 
for  the  USAF  to  use  in  the  future.  As  illustrated  in  Table  15, 
this  methodology  can  be  used  to  generate  a  "transferability  of 
skills  matrix"  for  enlisted  AFSs. 

The  second  major  conclusion  is  that  the  success  of  this 
research  project  provides  more  opportunities  for  the  use  of  this 
methodology  by  the  USAF.  We  have  covered  some  initial  ideas  in 
this  section,  however,  there  will  most  likely  be  other 
opportunities  to  extend  and  use  this  methodology  with  the 
enlisted  classification  system.  We  feel  highly  confident  in 
recommending  its  use  in  the  future. 
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p  1,  Background  Information 

Before  beginning  the  questionnaire,  please  fill  in  the  following  information  grids  on  the  front,  righthand  side  of  BOTH  answer 
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tasks  such  as  sweeping,  lifting,  cleaning,  sawing,  lubricating,  drilling, 
cutting,  hoisting,  chipping,  and  planing. 
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anticipate/figure  out/solve  problems,  and  plan  the  steps  and  procedures 
required  to  reach  a  solution  to  the  problem. 
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Performing  liquid  penetrant,  radiographic,  magnetic  particle, 
ultrasonic,  and  eddy  current  inspections;  Maintaining  nondestructive 
inspection  equipment. 
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automatic  direction  finder  systems* 
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DATA  RETRIEVAL  AND  STORAGE  SYSTEMS 
MAINTENANCE:  Maintaining  BRITE  II,  simulator  PIDP  systems, 
drum  units  and  plotters  (e.g.  XY  plotters). 
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step  3.  Overall  Ratings  of  Performance 
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69.  Supervisors  make  sure  that  new±y  rerramea  airmen  iwve  <-ne  ^ 

training  immediately. 

70.  Too  much  "red  tape"  frequently  keeps  me  from  getting  my  job  done  on  time. 
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80.  Supervisors  meet  regularly  with  newly  retrained  airmen  to  discuss  ways  to  apply 
training  on  the  job. 
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The  Air  Force  initiated  the  action,  and  I  did  not  want  to 


98.  Rate  the  training  you  received  in  technical  school  in  terms  of  how  well  it  prepared 
you  to  perform  in  your  current  (retraining)  specialty, 
a.  Not  applicable/I  did  not  attend  technical  school 
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You  will  note  that  you  have  the  questionnaire  as  well  as  two  green  "GENERAL  ANSWER  SHEET  TYPE  B*  forms.  All  of  your  answers 
are  to  be  made  on  these  forms  using  a  #2  pencil  so  that  we  can  more  rapidly  process  your  answers  into  our  research  data  base.  You  may, 
of  course,  complete  your  answer  first  on  the  questionnaire  itself  if  you  find  this  method  easier.  However,  your  answers  must  then.bg 
transferred  to  the  green  answer  sheets.  Specific  instructions  on  how  to  record  your  answers  will  be  included  with  each  part  of  the 
questionnaire.  Please  be  certain  to  read  the  instructions  for  each  part  of  this  questionnaire  carefully. 
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MANAGING  OTHERS:  Supervising  and  evaluating  others,  e.g. 
setting  goals,  coordinating  activities,  assigning  work,  evaluating 
performance,  conducting  meetings,  settling  conflicts. 
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DATA  RETRIEVAL  AND  STORAGE  SYSTEMS 
MAINTENANCE:  Maintaining  BRITE  II,  simulator  PIDP  systems, 
drum  units  and  plotters  (e.g.  XY  plotters).- 
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Never  meets  acceptable  i  Is  unable  to  complete  tasks  without  direct  I  I  -  Never  meets  acceptable  j  Is  uncooperative  and  ineffective  working  on  tasks  requiring 

level  of  proficiency  i  supervision  I  level  of  proficiency  j  teamwork;  unreceptive  to  guidance  by  supervisor  or 


Ill 


CD 

X 

P 


CD 

XX 

p 

c 

o 


CD 

X 

p 


CP 
G  P  CD 


* 

p 

o 


G 

0) 

Q) 

P  P 


-H  -H  OP  tJ'  O 
10 


P 

O 

x 

u 


cO  tO 

G>  to  a  G  <d  a) 

(0  to  g  x  Q) 

p  p 

•  P  CD  G  3 

C  C  -P  (D  C  fd 

O  CD  cd  P  O 
*H  g  -H  P 
(0  -p  <D  P  O  = 

a)  to  -p  ft 

(d  Oh: 

X  P  P  P  = 


2 

o 

>1  • 
G 

_  _  _  . .  _  P  O 

O  -H  10  a  (0  -H 

•H  10  &  P  P  XX  -P 

x  x  to  g  <o  p 

£  XX  o  Q)  g  t  G 

O  CO  CD  g  P  P 

o  -n  a)  x  a>  tj  g  -h 


p 

4J  -p  iH  (D 

to  r 

-p  s: 

co 

G  P 

PGP 

G  P 

O  X  X 

G  CD-H 

•H  10 

3  CD 

o 

d)  P  £ 
■P  P 

P  0 

G  0 

•n 

O 

X  G  P 

a)  .c 

O  X  P 

O 

(DOG 

S  P 

>iP 

G 

P 

0) 

to 

O 

CP 

0)  P  g 

G  X 

=  O 

P 

(0 

x:3H) 

to  -P 

-P 

P 

to 

-POO 

-H  TJ  (0 

G 

w 

•H 

p 

P  3= 

0)  o 

O 

a 

C  O'  3 

G  -H 

X 

-H  ««»•>►  CO 

0  CD 

-H  TJ 

P 

c 

P  CD 

CD 

O 

>i  O 

<0  C 

G 

u 

O  0) 

CD 

TJ  £1  P 

P 

P  *H 

O 

C/3 

G  P 

P 

0)  -h  3 

P  3 

P 

o 

a) 

Q)  tP 

tP 

p  p  o 

O  (0 

O 

o 

0)  0)  (0 

(0 

10  O  ><P  to 

•H  P 

to 

p 

s-i  <D  cn 

to 

o  to 

G  *H 

rH 

0 

H 

O'  k  -H 

•H 

P  Q)H 

0  TS 

CD  CD  X 

Eh 

< 

<  U'Q 

a 

<1)  TJ  0 

S  P 

•H 

*§ 

< 

-P 

TJ  3 

G 

P 

ft 

>i 

>i  >i  CD 

>1 

C  -P  -P 

G  O  P  to 

O 

rH 

rH  P  rH 

Q)  rH 

•HOC 

(0  >1  o 

(0 

CP 

4J  Q)  +J 

P 

CPI 

G  0 

c  a> 

0 

G 

0)  .C  JZ  £2 

CP  G 

Q)  -P 

C  <*-i 

X  *0 

O 

0)  O'-P  O' 

0 

P  to  X 

O  -H 

0 

P 

p  -H  -H  -H 

to 

p 

(0  0)  0) 

•H 

P  CD  P 

P 

tP  rH  CD  rH 

•H  p 

0 

p  ^  (0  to 

•H 

WCW2W 

a  w 

Q)  T>  0) 

(0  <D 

(0 

S  XX 

G  rH 

O  0 

to 

• 

•  •  •  • 

• 

• 

p  +j  +)  cur:  ^ 

(0 

c  OQ  a  Q  W  Pm  O 

>1  0 
m  w  m 

•H  g 
to  fff 

?  ft 

p 

— 

— 

_ i 

£>  P 
p  to  cd  x  <d 


X  to 

p 


<D  in 


G 

a) 


p 

to 

X 

•H 

*r^ 

P 

X 

VO 

CD 

id 

•H 

TJ 

O 

p  =*=  P 

g 

CD 

P 

G  P 

ft 

O 

CO 

•H 

X 

o 

•H 

C 

g  P 

rH 

p 

to 

CD 

G 

CD 

to 

a 

CD 

*• 

P 

• 

O 

P 

P 

TJ 

£ 

P  P 

£VH  X 

p 

0 

O 

G 

<D 

G 

O 

•H 

£ 

rH 

O 

0) 

P 

(0 

G 

G 

O 

CD 

TJ 

O 

CD 

D 

o 

i — 1 

X 

P 

to 

C 

4H 

•H 

CD 

G 

CD 

X 

* 

P  X 

CD 

O 

p 

CUV  P 

P 

o 

rH 

>i  CD 

CD 

0) 

•H 

O 

CD 

to 

(C 

£  TJ 

CD 

£ 

Ss 

to 

P 

O 

to 

to 

X 

O 

G 

to 

CD 

G  *0 

to 

••i 

to 

CD 

X 

rfl 

G 

CD 

•H 

g 

CP 

•H 

CO 

P 

P 

i 

X 

CD 

G 

P 

G 

0) 

CJ 

a 

p  p 

•H 

O 

CD 

X 

> 

CO 

o 

(0  4-> 

to 

CD 

P 

to 

to 

p 

G  P 

CO 

CD 

P 

•H 

G 

•H 

CQ 

H 

to 

(-1  TJ 

CP  £ 

(0 

TJ 

>i 

g 

p 

tn 

•H 

P 

G 

CD 

CD 

•H 

X 

3* 

> 

P 

rH 

O 

o 

0) 

P 

to 

to 

G 

CD 

0 

g 

p 

O 

P 

P 

- 

to 

• 

0 

CD 

TJ 

>n 

P 

X 

CD 

P 

0 

G 

•H 

P 

c0 

•H 

G 

rH 

0 

CO 

G 

G 

p 

P 

P 

0 

ft 

P 

P 

•H 

ft 

O 

P 

CO 

rH 

ft 

P 

H 

ft 

•H 

£ 

CD 

0 

TJ 

CD 

£ 

CD 

P 

G 

P 

X 

•H 

P 

O 

P 

to 

G 

to 

CD 

<D 

CD 

tr» 

CD 

> 

rX 

P 

c0 

cO 

G 

cd 

X 

g 

to 

O 

G 

>i 

TJ 

X 

CD 

rH 

G 

£ 

£ 

rH 

CO 

p 

CO 

to 

•H 

O 

p 

(0 

•H 

(0 

a) 

ft 

CO 

X 

TJ 

CD 

p 

CD 

P 

TJ 

0 

G 

•rH 

*H 

P 

G 

CO 

•H 

£ 

O 

P 

G 

CD 

ft 

P 

G 

G 

G 

0) 

P 

>i 

O 

*3 

g 

P 

G 

>i 

CD 

rH 

G 

G 

ft 

£ 

•H 

CD 

CD 

CD 

P 

T5 

G 

CD 

to 

iH 

•H 

TJ 

P 

ft 

rH 

G 

c0 

0 

(0 

X 

CD 

o 

P 

ft 

p 

X 

tP 

P 

CD 

• 

TJ 

•H 

P 

>1 

a) 

G 

£ 

G  rH 

•H 

•H 

to 

CD 

P 

rH 

P 

•H 

rH 

P 

CD 

c0 

rH 

•H 

O 

X 

-H 

to 

> 

<0  TJ 

a) 

g 

CD 

& 

P 

§ 

p 

to 

to 

g 

p 

cO 

G 

p 

•H 

0 

g 

o 

• 

to  • 

to 

tn 

0) 

P  to 

G 

•H 

G 

o 

G 

CD  g 

a) 

> 

•H 

CO 

O 

^  CD 

p 

P 

G 

rH  Tf 

P  rH 

4H 

CD 

•H 

O  X 

O 

ft  CO 

a)  £> 

s  p 

G 

P 

X 

O 

0  P 

H 

W  4-> 

H 

*r™> 

U  ft 

• 

• 

« 

• 

in 

VO 

r- 

CO 

VO 

VO 

vo 

VO 

112 


69.  I  frequently  have  trouble  getting  cooperation  from  others  who  are  supposed  to  help  me 
do  my  job. 

70.  Supervisors  meet  regularly  with  newly  retrained  airmen  to  discuss  ways  to  apply 
training  on  the  job. 
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TRANSFER  OF  SKILLS 


AIR  FORCE  MATERIEL  COMMAND 
BROOKS  AIR  FORCE  BASE,  TEXAS 


DEPARTMENT  OF  THE  AIR  FORCE 

ARMSTRONG  LABORATORY  (AFMC) 
BROOKS  AIR  FORCE  BASE.  TEXAS 
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Director 

Human  Resources  Directorate 


UNITED  STATES  AIR  FORCE 
TRANSFER  OF  SKILLS  QUESTIONNAIRE 
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be  certain  to  read  the  instructions  for  each  part  of  this  questionnaire  carefully. 


Background  Information 

Before  beginning  the  questionnaire,  fill  in  the  following  information  grids  on  the  front,  righthand  side  of  the  General  Answer  Sheet 
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ARTISTIC-AUDIO/VISUAL:  Tasks  such  as  designing/producing  photographs,  movies,  recordings, 
drawings;  playing  musical  instruments,  singing,  dancing,  acting;  operating  videotape  players,  slide 
projectors  etc. 
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CRYPTOGRAPHIC  EQUIPMENT  MAINTENANCE:  Maintaining  synchronizers  and  teletypewriter, 
cryptographic,  automatic  secure  voice  communications  fixed  record  communications  terminal 
equipment. 
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TAKE-OFF  AND  LANDING  CONTROL  SYSTEMS  MAINTENANCE:  Maintaining  RAPCON, 
video  mapper,  landing  control,  and  ASR,  PAR,  or  OPS  trailer  systems. 
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and  altimeters,  and  omnirange,  beacon,  instrument  landing,  long  range  navigation,  and  automatic 
direction  finder  systems. 
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APPENDIX  B 


REVISED  TAXONOMY  CATEGORY  DEFINITIONS  BASED  ON 
29  COMPONENT  SOLUTION 


CLERICAL  Tasks  such  as  filing,  preparing  forms,  answering 
telephones,  typing  reports,  and  proofreading.  Operating  office 
equipment  such  as  computers,  typewriters,  calculators,  dupli¬ 
cating  machines.  Processing  information  related  to  military  regu¬ 
lations,  federal  or  state  laws,  contracts,  and  legal  documents. 

PERSONNEL  Processing  data/information  about  individuals,  such  as 
employment  applications,  performance  reviews,  disciplinary 
reports,  media  releases,  production  records,  personnel  forecasts, 
training  records,  counseling  information,  and  social  services. 

MAINTAINING  INVENTORIES  AND  RECORDS  Maintaining 
materials/merchandise/  supplies/equipment  records.  Ordering, 
receiving,  maintaining,  routing  and  accounting  for  inventory. 
Preparing,  analyzing  and  maintaining  records  of  financial 
dealings,  property,  assets. 

MATHEMAT I CAL  Performing  numerical  operations  such  as  adding, 
subtracting,  multiplying,  dividing.  Computing  statistics  using 
formulas  and  equations.  Locating  statistics/data  in  graphs/ 
tables/charts .  Using  calculators  or  adding  machines  to  solve 
math  problems . 

PHYSICAL /MANUAL  LABOR  Nontechnical  manual  and  physical  tasks 
such  as  sweeping,  lifting,  carrying,  cleaning,  sawing,  lubrica¬ 
ting,  drilling,  cutting,  hoisting,  chipping,  and  planing.  Using 
basic  tools  such  as  a  hammer,  paint  scraper,  shovel,  or 
wheelbarrow. 

MANUFACTURING/FABRICATING  Making  things  from  materials  such  as 
sheet  metal,  metal  tubing,  glass,  brick,  plastic,  rubber,  paper, 
or  lumber.  For  example,  pressing,  mixing,  forging,  grinding, 
stitching,  forming,  melting,  or  chemically  treating  materials  to 
manufacture  things. 

CONSTRUCT I ON  Using  construction  information  such  as  materials 
lists,  building  designs,  etc.  Building/maintaining  structures 
made  of  brick,  stone,  lumber,  asphalt,  or  concrete,  such  as 
walls,  floors,  cabinets,  houses,  bridges,  towers,  roads,  or 
runways.  Laying/covering  with  roofing  materials,  floor 
coverings,  wall  paper. 

MEDICAL  -  PATIENT  CARE  Verbally  or  physically  interacting _ with 
patients,  e.g.,  bandaging,  giving  injections,  applying  medicines, 
drawing  blood,  performing  physical  therapy.  Reading/interpreting 
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medical  charts,  thermometer  readings,  test  results. 

MEDICAL  -  TECHNICAL  Performing  technical  procedures  in  a  medical 
lab  or  operating  room,  e.g.,  operating  X-ray  machine,  microscope, 
EKG  machine,  respirator,  ultrasound  machine. 

oral  AND  WRITTEN  COMMUNICATION  Reading/speaking/writing,  under¬ 
standing  words,  and  expressing  ideas,  including  receiving/proces¬ 
sing/initiating  letters,  books,  reports,  phone  calls,  orders, 
directions/instructions,  lectures,  contracts;  attending/conduc¬ 
ting  meetings,  making  presentations. 

PLANNING /PROBLEM  SOLVING  Using  available  information  to  antici¬ 
pate/figure  out/solve  problems,  and  plan  the  steps  and  procedures 
required  to  reach  a  solution  to  the  problem  (e.g.,  identifying  a 
traffic  problem  and  formulating  a  plan  for  rerouting  traffic) . 

SCIENCE  AND  ENGINEERING  Collecting/organizing/ summarizing 
technical  information,  or  information  about  people,  events, 
places.  Using  systematic/scientific  methods  to  test  theories/ 
products/equipment.  Writing  reports  of  results/findings.  Using 
technical  information  such  as  aerial  photos,  weather  forecasts, 
maps,  engineering  plans,  blueprints,  circuit  diagrams,  and  other 
designs/plans  for  equipment,  manufacturing  processes,  etc. 

ARTISTIC  -  AUDIO /VISUAL  Audio  and  visual  art  design/production, 

e.g.,  photography,  movies,  recordings,  drawings,  illustrations, 
layouts,  musical  compositions,  interior  decorations.  Performing, 
e.g.,  playing  musical  instruments,  singing,  dancing,  and  acting. 

Operating  equipment  such  as  turntables,  videotape  players,  slide 
projectors,  film  developing  equipment,  etc. 

FOOD  PREPARATION  Preparing/cooking  food,  using/producing  menus, 
recipes,  nutrition  guides,  food  requests  and  estimates. 

ANIMAL  CARE  Caring  for  animals,  including  grooming,  training, 
treating,  exercising,  or  tending  animals. 

FABRIC /ROPE  WORK  Sewing,  stitching,  threading,  weaving, 
combining,  or  separating  materials  such  as  fabric,  thread,  rope, 
material,  fiber,  and  string. 

MANAGING  OTHERS  Managing/administering/supervising/ evaluating 
others,  e.g.,  determining  goals  and  coordinating  others', 
activities,  assigning  work  to  others  and  supervising  their  work, 
evaluating  others'  performance,  making  staffing  decisions, 
conducting  group  meetings,  settling  conflicts  and  enforcing 
rules . 
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TRAINING  Explaining  ideas/procedures  to  others,  demonstrating 
how  a  task  is  done,  monitoring  learner  progress,  providing  feed¬ 
back  on  mistakes,  preparing  lesson  plans,  course  outlines,  etc. 

SURVEILLANCE  Using  codes/symbols  (e.g.,  traffic  control  "lingo," 
flag,  and  hand  signals) ,  detecting,  visualizing  and  recognizing 
objects  that  are  difficult  to  see  (e.g.,  ships  and  aircraft  at  a 
distance,  criminal  suspect  in  a  crowd) ,  tracking  and  pursuing 
moving  targets  or  objects,  using  firearms  or  other  handheld 
weapons,  enforcing  rules  or  laws.  \ 


ELECTRONIC  AND  MAINTENANCE  FACTORS 

CRYPTOGRAPHIC  EQUIPMENT  MAINTENANCE  Maintaining  synchronizers 
and  teletypwriter,  cryptographic,  automatic  secure  voice 
communications  fixed  record  communications  terminal  equipment. 

FLIGHT  CONTROL  AND  NAVIGATION  SYSTEMS  MAINTENANCE  Maintaining 
engine  and  integrated  navigation  and  flight  instruments,  and 
inertial  navigation,  fuel  savings  advisory,  automatic  flight 
control,  stability  augmentation  (damping) ,  quantity  indicating 
and  position  indicating  systems. 

MUNITIONS  HANDLING  Installing,  checking,  testing,  inspecting, 
maintaining,  assembling,  configuring,  and  reconditioning 
munitions  or  ordnance  devices.  Performing  munitions  control 
functions,  serviceability  inspections,  and  disposal  procedures, 
transporting,  storing,  handling,  and  shipping  munitions  and 
weapons . 

TEST.  MEASUREMENT.  AND  DIAGNOSTIC  EQUIPMENT  MAINTENANCE 
Maintaining  and  repairing  equipment  such  as  frequency  generating 
and  measuring,  electronic  precision  measuring,  waveform 
analyzing,  electro-mechanical, electrical  measurements  consoles, 
electrical  standards  sets,  radiac,  optical  precision  measuring, 
weapons  systems  precision  measuring,  and  voltage,  current,  and 
impedance  equipment . 

PNEUDRAULIC  SYSTEMS  INSPECTION  AND  MAINTENANCE  Installing, 
removing,  inspecting,  maintaining  and  performing  operational 
checks  of  hydraulic,  pneumatic,  and  pneudraulic  systems  and 
components . 

COMPUTER  SYSTEMS  MAINTENANCE  Maintaining,  installing  and 
modifying  keyboards,  printers,  central  processing  units,  magnetic 
tape  units,  video  display  equipment,  memory  assemblies, 
input/output  assemblies  (e.g.,  buffers,  controllers,  interfaces), 
magnetic  disk  systems  and  drum  units. 
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ground  based  communications  systems  maintenance  Maintaining 
receivers,  transmitters,  tranceivers,  antenna  systems,  ground 
radio  and  auxiliary  equipment,  and  communications -electronic 
equipment.  Note:  this  category  excludes  telephone  systems. 

NONDESTRUCTIVE  AIRCRAFT  STRUCTURE  INSPECTION  Performing  liquid 
penetrant,  radiographic,  magnetic  particle,  ultrasonic,  and  eddy 
current  inspections.  Maintaining  and  inspecting  nondestructive 
inspection  equipment.  \ 

AIRCRAFT  INSTRUMENTATION  MAINTENANCE  Preparing,  installing,  . 
calibrating,  aligning,  removing,  replacing,  inspecting,  testing, 
troubleshooting  and  maintaining  installed  aircraft 
instrumentation  equipment. 

METEOROLOGICAL  EQUIPMENT  MAINTENANCE  Maintaining  meteorological 
weather  radar,  nonelectronic  meteorological  instruments  and  solid 
state  barometers,  AN/FPS  systems,  and  wind,  temperature, 
visibility,  and  cloud  weather  equipment. 

AIRCRAFT  ARMAMENT  DELIVERY  AND  CONTROL  SYSTEMS  MAINTENANCE 
Inspecting,  maintaining,  and  performing  operational  or  functional 
checks  on  aircraft  installed  suspension,  launch,  and  release 
systems,  gun  systems,  aircraft  armament  systems,  and  support  and 
munitions  handling  equipment . 

TAKE-OFF  AND  LANDING  CONTROL  SYSTEMS  MAINTENANCE  Maintaining 
RAPCON,  video  mapper,  landing  control,  and  ASR,  PAR,  or  OPS 
trailer  systems. 

AIRCRAFT  NAVIGATION  AND  WEAPON  RADAR  MAINTENANCE  Installing, 
removing  and  maintaining  navigation,  radar,  radar  power 
distribution,  digital  modular  avionic,  and  weapons  release 
computer  systems . 

MALFUNCTION  AND  RECORDING  AND  CONTROL  SYSTEMS  MAINTENANCE 
Maintaining  malfunction  analysis  detection  and  recording  systems, 
general  purpose  or  navigational  computers,  and  digital  modular 
avionic  systems . 

TELEPHONE  SYSTEMS  MAINTENANCE  Installing,  repairing  and_ 
maintaining  key  telephone  systems,  telephone  systems  wiring  or 
components,  special  circuits,  and  mobile  or  fixed  switching 
center  equipment . 

AIRCRAFT  ELECTRICAL  SYSTEMS  MAINTENANCE  Inspecting,  maintaining 
and  isolating  malfunctions  in  aircraft  electrical  systems  and 
circuit  components . 
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AEROSPACE  GROUND  EQUIPMENT  CIRCUIT  COMPONENT  MAINTENANCE 
Maintaining  heating  and  refrigeration  systems  and . equipment 
coolers.  Maintaining  equipment  enclosures,  chassis,  drives,  and 
avionics  aerospace  ground  equipment  (AGE)  such  as  AGE  engines, 
motors  and  generators . 

AIR  AND  GROUND  BASED  NAVIGATIONAL  AIDS  MAINTENANCE  Maintaining 
transponders  and  radio/radar  altimeters,  and  VHF  omnirange, 
beacon,  variable  omnirange,  instrument  landing,  long  range v 
navigation,  and  automatic  direction  finder  systems. 

SATELLITE.  MISSILE.  AND  ANTENNA  SYSTEMS  INSPECTION  Preparing, 
installing,  and  maintaining  antenna,  missile,  and  satellite 
instrumentation  systems. 

AIRBORNE  TELEMETRY  MONITORING  AND  MAINTENANCE  Maintaining, 
monitoring,  and  operating  airborne  computer  and  switching 
systems,  synchronizers,  and  spread  spectrum  systems. 

SECURITY  PERIMETER  MAINTENANCE  Maintaining  solar  optical  and 
radio  observing  equipment,  radar  transit  surveillance  systems, 
and  base  and  installation  security  systems. 

AVIONICS  TEST  EQUIPMENT  MAINTENANCE  Maintaining  displays  and 
indicators,  computer  inertial,  and. radio  frequency  test  stations. 

Maintaining  test  equipment,  avionic  system  mockups,  aircraft 
mockups  and  manually  tested  equipment. 

QUALITY  CONTROL  Performing  quality  control,  quality  assurance, 
or  contract  monitor  functions. 

DATA  RETRIEVAL  AND  STORAGE  SYSTEMS  MAINTENANCE  Maintaining  BRITE 
II,  simulator,  PIDP  systems,  drum  units  and  plotters  (e.g.,  XY 
plotters) . 

SPACE -BASED  ASSETS  CONTROL  Maintaining  space  systems  equipment . 

Performing  maintenance/ job  control  or  sensor  technician 
functions . 

PROPULSION  AND  ENGINE  SYSTEMS  MAINTENANCE  Removal,  disassembly, 
inspection,  repair,  assembly,  and  installation  of  propulsion  and 
engine  systems  and  related  equipment. 


